-- Logs begin at Tue 2022-09-06 22:04:19 UTC, end at Tue 2022-09-06 23:02:38 UTC. -- Sep 06 22:28:10.602887 dv-cinder-852208-20-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 06 22:28:10.611628 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 6 22:28:10 2022] *** Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: nodename: dv-cinder-852208-20-iscsi Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: machine: x86_64 Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: clock source: unix Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: pcre jit disabled Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: detected number of CPU cores: 4 Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: current working directory: / Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: detected binary path: /usr/bin/uwsgi-core Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: your processes number limit is 47680 Sep 06 22:28:10.613569 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: your memory page size is 4096 bytes Sep 06 22:28:10.614182 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: detected max file descriptor number: 2048 Sep 06 22:28:10.614182 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: lock engine: pthread robust mutexes Sep 06 22:28:10.614182 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: thunder lock: enabled Sep 06 22:28:10.614182 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 06 22:28:10.614182 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 06 22:28:10.653894 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: Python main interpreter initialized at 0x55c558f699c0 Sep 06 22:28:10.653894 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: python threads support enabled Sep 06 22:28:10.653894 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: your server socket listen backlog is limited to 100 connections Sep 06 22:28:10.653894 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: your mercy for graceful operations on workers is 90 seconds Sep 06 22:28:10.654282 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: mapped 403077 bytes (393 KB) for 2 cores Sep 06 22:28:10.654362 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: *** Operational MODE: preforking *** Sep 06 22:28:10.654588 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: *** uWSGI is running in multiple interpreter mode *** Sep 06 22:28:10.654588 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: spawned uWSGI master process (pid: 81649) Sep 06 22:28:10.654797 dv-cinder-852208-20-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 06 22:28:10.656516 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: spawned uWSGI worker 1 (pid: 81665, cores: 1) Sep 06 22:28:10.656516 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: spawned uWSGI worker 2 (pid: 81666, cores: 1) Sep 06 22:28:10.656516 dv-cinder-852208-20-iscsi devstack@keystone.service[81649]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 06 22:28:14.934470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 22:28:14.934470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: from cryptography.utils import int_from_bytes Sep 06 22:28:14.935258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 22:28:14.935258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: from cryptography.utils import int_from_bytes Sep 06 22:28:14.935403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 22:28:14.935403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: from cryptography.utils import int_from_bytes Sep 06 22:28:14.935403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 06 22:28:14.935403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: from cryptography.utils import int_from_bytes Sep 06 22:28:14.984389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 22:28:14.984586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 22:28:14.984669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 06 22:28:14.984782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] command line args: [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 22:28:14.984885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 06 22:28:14.985033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 22:28:14.985126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] command line args: [] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 06 22:28:14.985205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ================================================================================ {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 22:28:14.985282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 06 22:28:14.985452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] admin_token = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.985531 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ================================================================================ {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 06 22:28:14.985698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] config_dir = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.985783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] admin_token = **** {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.985884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.985970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] config_dir = [] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.986067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] config_source = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.986145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.986247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] debug = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.986332 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] config_source = [] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.986446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.986738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] debug = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.986738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.986963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.987069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.987174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.987271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.987373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.987481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.987578 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.987691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.987799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.987901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] list_limit = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.987999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_config_append = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.988102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] log_config_append = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.988211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.988313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.988410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_dir = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.988505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] log_dir = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.988612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_file = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.988719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] log_file = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.988867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.988997 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.989122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.989209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.989281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.989409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.989490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.989570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.989688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.989765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.989889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.989971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990540 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.990999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.991146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.991235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.991994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.992077 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.992277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.992378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.992774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.992859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.39/identity {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.992992 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] publish_errors = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.993076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] publish_errors = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.993200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.993281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.993393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.993477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.993569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.993646 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.993814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.993916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.994044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.994121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.994277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] standard_threads = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.994354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] standard_threads = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.994506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.994592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.994744 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.994827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.994904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.994973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.995094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] use_journal = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.995167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] use_journal = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.995237 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] use_json = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.995353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] use_json = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.995427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] use_stderr = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.995504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] use_stderr = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.995587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] use_syslog = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.995664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] use_syslog = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.995765 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.995843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 06 22:28:14.996016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.996092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.996249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.996326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.996505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.996588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.996739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.996835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.997009 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.997088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.997243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.997326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.997487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.997568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.997720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.997797 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.997962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.998044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.998206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.998282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.998449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.998526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.998679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.998763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.998920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.999003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.999165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.999256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.999393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.999477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.999636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.999719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.999868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:14.999946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.000118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.000205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.000368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.000457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.000609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.000688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.000854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.000932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.001106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.001195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.001361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.001438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.001610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.001685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.001854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.001939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.002085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.002198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.002316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.002446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.002544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.002629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.002775 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.002858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.003018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.003094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.003262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.003348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.003487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.003563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.003714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.003797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.003955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.004040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.004196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.004273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.004428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.004503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.004671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.004774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.004921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.005000 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.005156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.005232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.005384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.005470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.005611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.005699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.005835 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.005911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.006062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.006215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.006294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.006365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.006510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.006587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.006756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.006843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.007011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.007099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.007246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.007348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.007878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.007994 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.008094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.008199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.008296 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.008391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.008564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.008681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.008866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.009006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.009092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.009326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.009445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.009562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.009798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.009876 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.010026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.010199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.010287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.010483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.010612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.010733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.010937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.011023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.011196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.011387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.011511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.011650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.011754 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.011887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.012122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.012207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.012390 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.012499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.012626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.012900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.012983 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.013143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.013267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.013378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.013613 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.013697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.013846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.014028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.014114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.014322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.014399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.014556 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.014738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.014818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.015021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.015098 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.015268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.015500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.015579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.015732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.015872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.015951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.016183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.016259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.016414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.016619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.016706 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.016922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.017010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.017172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.017367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.017446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.017633 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.017734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.017873 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.018106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.018183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.018340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.018475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.018555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.018786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.018865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.019015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.019205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.019290 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.019467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.019572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.019696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.019934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.020012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.020159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.020310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.020388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.020625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.020702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.020905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.021057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.021148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.021391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.021470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.021622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] auth.external = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.021771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.021849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.022133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.022220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.022365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.022573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.022657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] auth.password = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.022934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] auth.token = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.023019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.023174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.023320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.023405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.023641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.023718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.023870 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.024049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.024127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.024354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.024438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.024583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.024795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.024930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] credential.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.025076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.025156 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.025324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.025482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.025561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.025806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.025891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.026048 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.026186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.026265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.026513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.026597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.026764 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.026886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.026999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.027249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.027334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.027531 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.027619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.027773 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.027981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.028060 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.028259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.028336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.028512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.028731 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.028820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] federation.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.028991 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.029161 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.029240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.029467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.029546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.029694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.029882 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.external = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.029961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.030158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.030249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.030410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.030645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.030724 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.030897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.031039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.031127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.031362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.031440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.password = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.031596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.031737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] auth.token = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.031841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.032086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.032165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.032321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.032461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.032539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.032806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.032901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.033038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.033213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.033293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.033507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.033585 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.033756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.033966 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.034046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.034207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.034322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.034434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.034674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.034751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.034917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.035054 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.035160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.035411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.035492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.035654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.035804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.035920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.036154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.036240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.036388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.036548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.036626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.036863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.036943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.037097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.037311 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.037391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.037545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.037704 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.037784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.038013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.038092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.038756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.039036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.039367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.039625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.039894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.040142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.040404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.040642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.040903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.041159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.041460 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.041546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.041730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.041836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.041981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.042089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.042237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.042342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.042489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.042594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.042726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.042838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.042969 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.043127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.043234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.043340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.043444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.043547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.043661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.043767 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.043878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.043983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.044113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.044222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.044336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.044442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.044590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.044708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.044865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.044998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.045154 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.045253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.045395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.045501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.045645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.045750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.045896 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.046002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.046126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.046230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.046358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.046470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.046587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.046695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.046800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.046905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.047051 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.047165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.047284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.047390 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.047534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.047656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.047793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.047882 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.048013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.048191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.048273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.048481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.048558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.048713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.048934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.049017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.049193 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.049271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.049427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.049638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.049719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.049892 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.049970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.050138 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.050335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.050422 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.050616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.050694 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.050863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.051061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.051141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.051327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.051442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.051562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.051790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.051869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.052025 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.052166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.052247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.052489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.052566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.052737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.052914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.053006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.053238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.053316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.053467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.053648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.053726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.054542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.054645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.054778 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.054999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.055189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.055388 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.055476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.055670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.055793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.055901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.056037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.056141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.056283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.056390 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] resource.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.056520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.056624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.056804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.056913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.057039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.057149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.057319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.057431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.057567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.057683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.057800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.057910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.058016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.058122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.058262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.058376 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.058491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.058592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.058717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] role.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.058844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.058965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] role.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.059068 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.059198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.059303 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.059446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.059550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.059693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.059808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.059928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.060035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.060141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.060249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.060389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.060502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.060620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.060729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.060871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.061004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.061127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.061206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.061359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.061533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.061613 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.061839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.061917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.062087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.062239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.062318 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.062592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.062731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.062967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.063058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.063215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.063341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.063451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.063701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.063780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.063948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.064077 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.064180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.064432 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.064509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.064683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.064864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.064943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.065164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.065243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.065396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.065522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.065629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.065879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.065958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.066133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.066242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.066379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.066597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.066678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.066843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.066971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.067073 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.067308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] token.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.067394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.067543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.067727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.067805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.068011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.068153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.068266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.068514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.068630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.068730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.068870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.069029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.069149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.069295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.069403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.069569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.069661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.069817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.069930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.070123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.070237 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.070346 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.070456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.070589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.070714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.070864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.070993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.071100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.071215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.071327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.071428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.071573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.071679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.071832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.connection = **** {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.071936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.072074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.072180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.072319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.072433 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.072542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.072649 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.072793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.072906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.073032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.073144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.073252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.073360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] role.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.073467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.073581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] role.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.073719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.073801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.073911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.074084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.074164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.074371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.074457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.074604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.074825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.074906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.075069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.075206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.075295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.075525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.075604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.075764 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.075942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.076022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.076152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81666) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 22:28:15.076319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.076693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.077075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.077454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.077828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.078205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.078578 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.079198 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.079564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.079944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.080361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.080799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.081223 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.081464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.081721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.081975 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.082214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.082450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.082683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.082916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.083167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.083421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.083712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.083950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.084185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.084416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.084651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.084914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.085216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.085513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.085879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.086243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.086610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.086988 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.087362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.087741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.088102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.088464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.088850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.089231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.089648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.089997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.090368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.090769 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.091166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.connection = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.091542 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.091922 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.092281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.092678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.093114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.093501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.093848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.094193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.094553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.094853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.095100 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.095338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.095587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.095823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.096067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.096299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.096539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.096798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 06 22:28:15.096965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81665) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 06 22:28:15.105953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.106182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.106404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.114641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.122538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.122763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.122977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.131172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.143638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.143864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.150581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.150810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.197451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.197839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.245586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.245813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.245945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.246178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.409968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.410241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.410378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.410513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.410672 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.410806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.410934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.411061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.411188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.411323 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.411450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.411450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.435575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.435806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.436037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.436268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.436399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.436528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.436655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.436802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.436954 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.437082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.437220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.437347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.437471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.437651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.449159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.449307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.461853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.462084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.462244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.462389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.462533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.462661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.462787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.462913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.463049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.463183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.463183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81665) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.463427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.463427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.463640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81666) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 06 22:28:15.479738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.479946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 22:28:15.480287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.480488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 22:28:15.480789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.481050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.481304 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.481556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.482124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.482337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 22:28:15.482681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.482883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 06 22:28:15.483161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.483425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.483686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.483944 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.491036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.491983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.492257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.492541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.494589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.494871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.495142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.495426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.499526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.502419 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.502561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.502714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.502996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.503277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.503560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.503823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.504090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.504359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.504624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.504928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.505194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.505361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.505650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.505937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.506223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.506510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.506774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.507986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.508379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.508673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.508980 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.509257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.529392 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.530846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.531134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.532487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.533984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.534272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.535892 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.536175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.536451 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.539462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.539751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.540036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.543063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.543347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.546446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.546735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.548998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.549297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.549586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.549874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.550153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.550431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.550707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.550976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.552476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.554988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.559082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.559384 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.559676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.559968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.560254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.560533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.567554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.567836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.571807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.574998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.575293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.576404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.576699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.579462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.580584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.582554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.583816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.584111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.586876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.587167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.587446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.587723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.587999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.588275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.589076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.592295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.595560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.595858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.596140 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.596418 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.596691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.597002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.604679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.604989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.605258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.605520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.612389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.613888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.613972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.614176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.614443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.614703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.615471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.615744 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.618459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.618740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.619018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.619299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.619696 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.621602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.623137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.624662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.624959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.627676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.627959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.628240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.628536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.628927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.630882 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.631163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.631439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.631714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.632232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.632781 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.633073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.640348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.640631 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.640938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.641217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.641595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.641874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.642161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.649114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.652030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.654940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.656379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.657985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.658072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.658268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.658660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.660975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.664444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.665686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.665945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.667159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.667406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.667687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.668085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.670083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.670476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.670761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.671036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.674491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.675952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.678916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.679078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.679259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.679357 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.679546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.679631 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.679885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.686458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81665) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.686743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.687024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.687299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.687573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.687793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.687876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.688099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.688186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.688379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.688461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.688656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.688959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.689443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.689569 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.689746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.690016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81665) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.695920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81666) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 06 22:28:15.696200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.696479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.696774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.697049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.697319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.697592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.697877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.698154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.698430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.698705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.698962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.699268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.699505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81666) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 06 22:28:15.733204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c558f699c0 pid: 81665 (default app) Sep 06 22:28:15.735205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55c558f699c0 pid: 81666 (default app) Sep 06 22:28:15.737034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1298351-6f84-4148-8ff8-dffbbc3b9ee8 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:15.737320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1298351-6f84-4148-8ff8-dffbbc3b9ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:15.737589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1298351-6f84-4148-8ff8-dffbbc3b9ee8 None None] PATH_INFO: `/v3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:15.741735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1/1] 10.222.0.39 () {50 vars in 791 bytes} [Tue Sep 6 22:28:15 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:22.525552 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad1cf5-18ec-4de4-ad63-abf160bb0b7e None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:22.526062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad1cf5-18ec-4de4-ad63-abf160bb0b7e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:22.526169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ad1cf5-18ec-4de4-ad63-abf160bb0b7e None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:22.530334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1/2] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:22 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:22.535858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d37c32-6432-4701-8601-4509c01c7def None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:22.536404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d37c32-6432-4701-8601-4509c01c7def None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:22.536404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d37c32-6432-4701-8601-4509c01c7def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:22.608195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: INFO dbcounter [None req-e1d37c32-6432-4701-8601-4509c01c7def None None] Registered counter for database keystone Sep 06 22:28:22.627104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG oslo_db.sqlalchemy.engines [None req-e1d37c32-6432-4701-8601-4509c01c7def None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81665) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 22:28:22.633954 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writer thread running {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 22:28:22.811288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG passlib.handlers.bcrypt [None req-e1d37c32-6432-4701-8601-4509c01c7def None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81665) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 22:28:22.811594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG passlib.handlers.bcrypt [None req-e1d37c32-6432-4701-8601-4509c01c7def None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81665) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 22:28:22.869593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e1d37c32-6432-4701-8601-4509c01c7def None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:22.871202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e1d37c32-6432-4701-8601-4509c01c7def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:22.999274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 2/3] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 464 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Sep 06 22:28:23.419860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad1cf5-18ec-4de4-ad63-abf160bb0b7e None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:23.488874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: INFO dbcounter [None req-e1ad1cf5-18ec-4de4-ad63-abf160bb0b7e None None] Registered counter for database keystone Sep 06 22:28:23.510643 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG oslo_db.sqlalchemy.engines [None req-e1ad1cf5-18ec-4de4-ad63-abf160bb0b7e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81666) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 06 22:28:23.516822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writer thread running {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 06 22:28:23.634810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ad1cf5-18ec-4de4-ad63-abf160bb0b7e None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:23.639585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb74fc-deef-4947-af0b-de9ce797989f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:23.640206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb74fc-deef-4947-af0b-de9ce797989f admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:23.640481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb74fc-deef-4947-af0b-de9ce797989f admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:23.640750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb74fc-deef-4947-af0b-de9ce797989f admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:23.649715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb74fc-deef-4947-af0b-de9ce797989f admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:23.701955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2fdb74fc-deef-4947-af0b-de9ce797989f admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 22:28:23.702234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb74fc-deef-4947-af0b-de9ce797989f admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:23.712013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 2/4] 10.222.0.39 () {58 vars in 1324 bytes} [Tue Sep 6 22:28:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:28:23.717938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d37c32-6432-4701-8601-4509c01c7def None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:23.733076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d37c32-6432-4701-8601-4509c01c7def None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:23.737483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800e819-d7e9-4929-b547-8cbff088f280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:23.738580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e819-d7e9-4929-b547-8cbff088f280 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:23.738857 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e819-d7e9-4929-b547-8cbff088f280 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:23.739127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800e819-d7e9-4929-b547-8cbff088f280 admin admin] PATH_INFO: `/v3/projects/693d7b3554de4d2c9e0e732722fe61d1` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:23.740706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e819-d7e9-4929-b547-8cbff088f280 admin admin] RBAC: Authorizing `identity:get_project(project_id=693d7b3554de4d2c9e0e732722fe61d1)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:23.794179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: WARNING keystone.common.rbac_enforcer.enforcer [None req-2800e819-d7e9-4929-b547-8cbff088f280 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 06 22:28:23.794525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2800e819-d7e9-4929-b547-8cbff088f280 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:23.799061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 3/5] 10.222.0.39 () {56 vars in 1207 bytes} [Tue Sep 6 22:28:23 2022] GET /identity/v3/projects/693d7b3554de4d2c9e0e732722fe61d1 => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:25.908189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cac170-f505-4775-8b42-463c81d983d7 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:25.908722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cac170-f505-4775-8b42-463c81d983d7 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:25.908722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cac170-f505-4775-8b42-463c81d983d7 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:25.913418 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 3/6] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:25.918759 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105ba89-02da-44cb-ba88-a846f945f076 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:25.919177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105ba89-02da-44cb-ba88-a846f945f076 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:25.919343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105ba89-02da-44cb-ba88-a846f945f076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:25.939298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3105ba89-02da-44cb-ba88-a846f945f076 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:25.940825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3105ba89-02da-44cb-ba88-a846f945f076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:25.949162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 4/7] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 06 22:28:26.214960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cac170-f505-4775-8b42-463c81d983d7 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:26.227163 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cac170-f505-4775-8b42-463c81d983d7 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:26.231314 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8163608-42f7-4bc2-bacd-e766bc2013b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:26.231906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8163608-42f7-4bc2-bacd-e766bc2013b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:26.232184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8163608-42f7-4bc2-bacd-e766bc2013b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:26.232454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8163608-42f7-4bc2-bacd-e766bc2013b9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:26.239938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8163608-42f7-4bc2-bacd-e766bc2013b9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:26.241929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8163608-42f7-4bc2-bacd-e766bc2013b9 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:26.250517 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 4/8] 10.222.0.39 () {58 vars in 1324 bytes} [Tue Sep 6 22:28:26 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:28:26.255792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105ba89-02da-44cb-ba88-a846f945f076 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:26.265887 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105ba89-02da-44cb-ba88-a846f945f076 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:26.270235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea556d8-e61b-4189-aa0b-4b8d8ae93ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:26.270861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea556d8-e61b-4189-aa0b-4b8d8ae93ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:26.271139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea556d8-e61b-4189-aa0b-4b8d8ae93ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:26.271411 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea556d8-e61b-4189-aa0b-4b8d8ae93ca8 admin admin] PATH_INFO: `/v3/users/fee2283327204c6f8ee811e2c566e379` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:26.273357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea556d8-e61b-4189-aa0b-4b8d8ae93ca8 admin admin] RBAC: Authorizing `identity:get_user(user_id=fee2283327204c6f8ee811e2c566e379)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:26.275160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea556d8-e61b-4189-aa0b-4b8d8ae93ca8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:26.276715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 5/9] 10.222.0.39 () {56 vars in 1198 bytes} [Tue Sep 6 22:28:26 2022] GET /identity/v3/users/fee2283327204c6f8ee811e2c566e379 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:28.521300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70d259-6ed2-4a70-9470-8d5488a83d5c None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:28.521833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70d259-6ed2-4a70-9470-8d5488a83d5c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:28.521833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f70d259-6ed2-4a70-9470-8d5488a83d5c None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:28.523405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 5/10] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:28.528593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674735c8-2b4c-47cf-9023-fb94494e91d7 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:28.529034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674735c8-2b4c-47cf-9023-fb94494e91d7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:28.529171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674735c8-2b4c-47cf-9023-fb94494e91d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:28.544717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe029-c275-459a-8d2d-4ff5f4a66597 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:28.545029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe029-c275-459a-8d2d-4ff5f4a66597 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:28.545302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fe029-c275-459a-8d2d-4ff5f4a66597 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:28.546583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 6/11] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:28:28.548122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-674735c8-2b4c-47cf-9023-fb94494e91d7 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:28.549355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-674735c8-2b4c-47cf-9023-fb94494e91d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:28.551708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd938d8-b758-43bb-9873-15c054cc1223 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:28.551989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd938d8-b758-43bb-9873-15c054cc1223 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:28.552254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd938d8-b758-43bb-9873-15c054cc1223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:28.594825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG passlib.handlers.bcrypt [None req-ecd938d8-b758-43bb-9873-15c054cc1223 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81666) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 06 22:28:28.595216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG passlib.handlers.bcrypt [None req-ecd938d8-b758-43bb-9873-15c054cc1223 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81666) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 06 22:28:28.630249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 6/12] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:28.646099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-ecd938d8-b758-43bb-9873-15c054cc1223 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:28.648745 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ecd938d8-b758-43bb-9873-15c054cc1223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:28.748614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 7/13] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:28:28.896284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674735c8-2b4c-47cf-9023-fb94494e91d7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:28.921357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:29.003258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674735c8-2b4c-47cf-9023-fb94494e91d7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:29.024217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd938d8-b758-43bb-9873-15c054cc1223 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:29.037695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016cb92-685d-48d2-b387-6f3f340d568e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:29.038356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016cb92-685d-48d2-b387-6f3f340d568e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:29.038649 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016cb92-685d-48d2-b387-6f3f340d568e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:29.038901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f016cb92-685d-48d2-b387-6f3f340d568e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:29.105544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016cb92-685d-48d2-b387-6f3f340d568e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:29.107004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f016cb92-685d-48d2-b387-6f3f340d568e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:29.123435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd938d8-b758-43bb-9873-15c054cc1223 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:29.125747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:29.129617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:29.162303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef986cbf-d1f6-4c9f-8696-72005349274c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:29.163606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef986cbf-d1f6-4c9f-8696-72005349274c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:29.163884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef986cbf-d1f6-4c9f-8696-72005349274c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:29.164151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef986cbf-d1f6-4c9f-8696-72005349274c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:29.176950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef986cbf-d1f6-4c9f-8696-72005349274c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-ef986cbf-d1f6-4c9f-8696-72005349274c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:29.187405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:29.188443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:28:29.189442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 8/14] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:28:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:28:29.194880 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef986cbf-d1f6-4c9f-8696-72005349274c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:29.207247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 7/15] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:28:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:28:29.213096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016cb92-685d-48d2-b387-6f3f340d568e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:29.294126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef986cbf-d1f6-4c9f-8696-72005349274c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:29.311136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f016cb92-685d-48d2-b387-6f3f340d568e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:29.328734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76684e-3c68-4dae-aa7a-83de59ef8caf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:29.329382 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76684e-3c68-4dae-aa7a-83de59ef8caf None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:29.329655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76684e-3c68-4dae-aa7a-83de59ef8caf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:29.329920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d76684e-3c68-4dae-aa7a-83de59ef8caf None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:29.330572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76684e-3c68-4dae-aa7a-83de59ef8caf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:29.330867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76684e-3c68-4dae-aa7a-83de59ef8caf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:29.332274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d76684e-3c68-4dae-aa7a-83de59ef8caf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:29.333652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:29.344773 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 9/16] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:29.347249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eadf03-f0f7-4025-9dab-8260231a230e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:29.347882 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eadf03-f0f7-4025-9dab-8260231a230e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:29.348157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eadf03-f0f7-4025-9dab-8260231a230e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:29.348426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eadf03-f0f7-4025-9dab-8260231a230e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:29.351128 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76684e-3c68-4dae-aa7a-83de59ef8caf None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:29.358928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eadf03-f0f7-4025-9dab-8260231a230e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:29.360393 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eadf03-f0f7-4025-9dab-8260231a230e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:29.377837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-56eadf03-f0f7-4025-9dab-8260231a230e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:28:29.379084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:29.380588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:28:29.382462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:28:29.382462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:28:29.382462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:28:29.382462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 8/17] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:28:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:28:29.386509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eadf03-f0f7-4025-9dab-8260231a230e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:29.455361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d76684e-3c68-4dae-aa7a-83de59ef8caf None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:29.485877 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eadf03-f0f7-4025-9dab-8260231a230e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:29.488936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945ca13-7163-465c-9183-82f8a917b6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:29.489565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945ca13-7163-465c-9183-82f8a917b6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:29.489841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945ca13-7163-465c-9183-82f8a917b6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:29.490111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a945ca13-7163-465c-9183-82f8a917b6a3 None admin] PATH_INFO: `/v3/users/fee2283327204c6f8ee811e2c566e379` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:29.492685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945ca13-7163-465c-9183-82f8a917b6a3 None admin] RBAC: Authorizing `identity:get_user(user_id=fee2283327204c6f8ee811e2c566e379)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:29.494157 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a945ca13-7163-465c-9183-82f8a917b6a3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:29.495706 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 10/18] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:29 2022] GET /identity/v3/users/fee2283327204c6f8ee811e2c566e379 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:29.501104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945ca13-7163-465c-9183-82f8a917b6a3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:29.519196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18c4aa-53fe-40e7-838d-e0a0614a8a2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:29.519810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18c4aa-53fe-40e7-838d-e0a0614a8a2d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:29.520087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18c4aa-53fe-40e7-838d-e0a0614a8a2d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:29.520359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18c4aa-53fe-40e7-838d-e0a0614a8a2d None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:29.521027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18c4aa-53fe-40e7-838d-e0a0614a8a2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:29.521330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18c4aa-53fe-40e7-838d-e0a0614a8a2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:29.522734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18c4aa-53fe-40e7-838d-e0a0614a8a2d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:29.540663 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 9/19] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:28:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:29.599809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:29.608205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a945ca13-7163-465c-9183-82f8a917b6a3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:29.644049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075496e3-c2a9-49de-8811-e3859bc9c78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:29.644691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075496e3-c2a9-49de-8811-e3859bc9c78a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:29.644998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075496e3-c2a9-49de-8811-e3859bc9c78a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:29.645271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075496e3-c2a9-49de-8811-e3859bc9c78a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:29.646425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075496e3-c2a9-49de-8811-e3859bc9c78a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:29.648392 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075496e3-c2a9-49de-8811-e3859bc9c78a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:29.650010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 11/20] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:28:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:29.656029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18c4aa-53fe-40e7-838d-e0a0614a8a2d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:29.693399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:29.764636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18c4aa-53fe-40e7-838d-e0a0614a8a2d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:29.800143 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ebd310-3e37-4f10-a6cf-b39513994f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:29.800751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebd310-3e37-4f10-a6cf-b39513994f35 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:29.801063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebd310-3e37-4f10-a6cf-b39513994f35 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:29.801329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ebd310-3e37-4f10-a6cf-b39513994f35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:29.802051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebd310-3e37-4f10-a6cf-b39513994f35 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.domain.id=default, user.id=fee2283327204c6f8ee811e2c566e379) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:29.802352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebd310-3e37-4f10-a6cf-b39513994f35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:29.803898 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ebd310-3e37-4f10-a6cf-b39513994f35 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:29.810707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 10/21] 10.222.0.39 () {56 vars in 1322 bytes} [Tue Sep 6 22:28:29 2022] GET /identity/v3/role_assignments?user.id=fee2283327204c6f8ee811e2c566e379&scope.domain.id=default&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 233 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:32.203902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd11cb-d8ef-412b-94a9-ccf0ae895544 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:32.204742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd11cb-d8ef-412b-94a9-ccf0ae895544 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:32.205176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd11cb-d8ef-412b-94a9-ccf0ae895544 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:32.206649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 12/22] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:32.222265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01697541-01d7-4a15-b48f-b9d03d91bfe4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:32.223089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01697541-01d7-4a15-b48f-b9d03d91bfe4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:32.223507 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01697541-01d7-4a15-b48f-b9d03d91bfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:32.249870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-01697541-01d7-4a15-b48f-b9d03d91bfe4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:32.251093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-01697541-01d7-4a15-b48f-b9d03d91bfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:32.350692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 11/23] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:32.448507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56757e5d-0934-4b0f-9303-ecf34c20b09d None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:32.449173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56757e5d-0934-4b0f-9303-ecf34c20b09d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:32.449643 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56757e5d-0934-4b0f-9303-ecf34c20b09d None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:32.451360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 13/24] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:32.456901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce0d2-33c3-41ea-a2d8-88f76fca62e4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:32.457472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce0d2-33c3-41ea-a2d8-88f76fca62e4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:32.457957 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ce0d2-33c3-41ea-a2d8-88f76fca62e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:32.485788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f46ce0d2-33c3-41ea-a2d8-88f76fca62e4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:32.487313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f46ce0d2-33c3-41ea-a2d8-88f76fca62e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:32.515355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:32.580784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 12/25] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:28:32.638934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbe625-23b5-47b7-8537-ee5cc9da668c None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:32.639437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbe625-23b5-47b7-8537-ee5cc9da668c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:32.639850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cbe625-23b5-47b7-8537-ee5cc9da668c None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:32.641345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 14/26] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:32.646618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a9cfa-639b-4fc1-86c4-39a958b2a079 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:32.647691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a9cfa-639b-4fc1-86c4-39a958b2a079 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:32.648268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204a9cfa-639b-4fc1-86c4-39a958b2a079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:32.674925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-204a9cfa-639b-4fc1-86c4-39a958b2a079 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:32.676332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-204a9cfa-639b-4fc1-86c4-39a958b2a079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:32.772413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 13/27] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:32.778437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbe625-23b5-47b7-8537-ee5cc9da668c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:32.930449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cbe625-23b5-47b7-8537-ee5cc9da668c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:32.972598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b372af-c968-4b92-811a-15cb99431001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:32.973702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b372af-c968-4b92-811a-15cb99431001 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:32.974144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b372af-c968-4b92-811a-15cb99431001 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:32.974542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b372af-c968-4b92-811a-15cb99431001 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:33.002838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b372af-c968-4b92-811a-15cb99431001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:33.004292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b372af-c968-4b92-811a-15cb99431001 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:33.013896 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2958b8e-74a9-413f-85f4-7a769782276f None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:33.014177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2958b8e-74a9-413f-85f4-7a769782276f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:33.014627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2958b8e-74a9-413f-85f4-7a769782276f None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:33.016087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 14/28] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-e4b372af-c968-4b92-811a-15cb99431001 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:28:33.025427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:33.026459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:28:33.027463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:28:33.027463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:28:33.027463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:28:33.027463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 15/29] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:28:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:33.033181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be0f2a-e4a9-44be-84e6-fe0a9f843d95 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:33.033311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b372af-c968-4b92-811a-15cb99431001 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:33.033715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be0f2a-e4a9-44be-84e6-fe0a9f843d95 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:33.034154 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be0f2a-e4a9-44be-84e6-fe0a9f843d95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:33.065074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:33.077099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-51be0f2a-e4a9-44be-84e6-fe0a9f843d95 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:33.078291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-51be0f2a-e4a9-44be-84e6-fe0a9f843d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:33.187232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b372af-c968-4b92-811a-15cb99431001 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:33.195742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 15/30] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:33.200850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be0f2a-e4a9-44be-84e6-fe0a9f843d95 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:33.235506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:33.239237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d18eb1-61c5-4f24-ae98-b7b748d80a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:33.239931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d18eb1-61c5-4f24-ae98-b7b748d80a15 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:33.240213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d18eb1-61c5-4f24-ae98-b7b748d80a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:33.240593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d18eb1-61c5-4f24-ae98-b7b748d80a15 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:33.241492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d18eb1-61c5-4f24-ae98-b7b748d80a15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:33.242350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d18eb1-61c5-4f24-ae98-b7b748d80a15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:33.243951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d18eb1-61c5-4f24-ae98-b7b748d80a15 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:33.277965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 16/31] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:28:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:33.279058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d18eb1-61c5-4f24-ae98-b7b748d80a15 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:33.390235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be0f2a-e4a9-44be-84e6-fe0a9f843d95 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:33.417162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:33.439155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b631243-3ab0-4141-bbb1-fc03c2c1c032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:33.440135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b631243-3ab0-4141-bbb1-fc03c2c1c032 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:33.440621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b631243-3ab0-4141-bbb1-fc03c2c1c032 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:33.441097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b631243-3ab0-4141-bbb1-fc03c2c1c032 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:33.453219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b631243-3ab0-4141-bbb1-fc03c2c1c032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-6b631243-3ab0-4141-bbb1-fc03c2c1c032 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:33.460678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:33.461647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:33.462373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:28:33.464664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 16/32] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:28:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:33.465587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b631243-3ab0-4141-bbb1-fc03c2c1c032 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:33.475247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d18eb1-61c5-4f24-ae98-b7b748d80a15 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:33.511515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac40a1-a01a-413b-ad7a-bee0c0c8723b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:33.512130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac40a1-a01a-413b-ad7a-bee0c0c8723b None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:33.512403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac40a1-a01a-413b-ad7a-bee0c0c8723b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:33.512959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac40a1-a01a-413b-ad7a-bee0c0c8723b None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:33.513662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac40a1-a01a-413b-ad7a-bee0c0c8723b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:33.515898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac40a1-a01a-413b-ad7a-bee0c0c8723b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:33.579854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:33.604416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b631243-3ab0-4141-bbb1-fc03c2c1c032 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:33.639613 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f4aa49-1481-4d56-891a-d559435612c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:33.640219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4aa49-1481-4d56-891a-d559435612c0 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:33.640492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4aa49-1481-4d56-891a-d559435612c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:33.640777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f4aa49-1481-4d56-891a-d559435612c0 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:33.641432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4aa49-1481-4d56-891a-d559435612c0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:33.642902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f4aa49-1481-4d56-891a-d559435612c0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:33.652900 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 17/33] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Sep 6 22:28:33 2022] POST /identity/v3/roles => generated 219 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:28:33.654802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac40a1-a01a-413b-ad7a-bee0c0c8723b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:33.787066 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac40a1-a01a-413b-ad7a-bee0c0c8723b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:33.832815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343dffa-23cb-459c-b453-0ca0d938242e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:33.832815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343dffa-23cb-459c-b453-0ca0d938242e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:33.833049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343dffa-23cb-459c-b453-0ca0d938242e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:33.833350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5343dffa-23cb-459c-b453-0ca0d938242e None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:33.833933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343dffa-23cb-459c-b453-0ca0d938242e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:33.834376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343dffa-23cb-459c-b453-0ca0d938242e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:33.836653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5343dffa-23cb-459c-b453-0ca0d938242e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:33.838463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:33.847156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 18/34] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:33.853848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343dffa-23cb-459c-b453-0ca0d938242e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:33.930465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 17/35] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:28:33 2022] POST /identity/v3/projects => generated 301 bytes in 465 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:28:33.942444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9824-509a-483b-aa29-557c9d794489 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:33.942722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9824-509a-483b-aa29-557c9d794489 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:33.943000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033e9824-509a-483b-aa29-557c9d794489 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:33.948805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 18/36] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:33 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:33.948805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e9824-509a-483b-aa29-557c9d794489 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:33.995178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5343dffa-23cb-459c-b453-0ca0d938242e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:34.046292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b30152-526c-410d-b686-e1fe6c1962da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:34.047258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b30152-526c-410d-b686-e1fe6c1962da None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:34.047709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b30152-526c-410d-b686-e1fe6c1962da None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:34.048271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b30152-526c-410d-b686-e1fe6c1962da None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:34.049517 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b30152-526c-410d-b686-e1fe6c1962da None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:34.051071 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b30152-526c-410d-b686-e1fe6c1962da None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:34.070504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033e9824-509a-483b-aa29-557c9d794489 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:34.073135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 19/37] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Sep 6 22:28:33 2022] POST /identity/v3/roles => generated 225 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:28:34.075833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf776dc9-e59e-460c-83ee-e977785dc5cc None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:34.076276 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf776dc9-e59e-460c-83ee-e977785dc5cc None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:34.076689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf776dc9-e59e-460c-83ee-e977785dc5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:34.104360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-bf776dc9-e59e-460c-83ee-e977785dc5cc None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:34.107041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-bf776dc9-e59e-460c-83ee-e977785dc5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:34.109274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:34.119683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458964d-1f0b-4c65-b462-cf608505ab43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:34.120803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458964d-1f0b-4c65-b462-cf608505ab43 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:34.121374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458964d-1f0b-4c65-b462-cf608505ab43 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:34.122043 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458964d-1f0b-4c65-b462-cf608505ab43 None admin] PATH_INFO: `/v3/users/fee2283327204c6f8ee811e2c566e379` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:34.123941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458964d-1f0b-4c65-b462-cf608505ab43 None admin] RBAC: Authorizing `identity:get_user(user_id=fee2283327204c6f8ee811e2c566e379)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:34.126091 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458964d-1f0b-4c65-b462-cf608505ab43 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:34.128369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 19/38] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:33 2022] GET /identity/v3/users/fee2283327204c6f8ee811e2c566e379 => generated 247 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:34.145257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458964d-1f0b-4c65-b462-cf608505ab43 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:34.188932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:34.211870 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 20/39] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:34.235845 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f514b-a7bc-4303-a377-053690797daa None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:34.235845 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f514b-a7bc-4303-a377-053690797daa None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:34.236082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135f514b-a7bc-4303-a377-053690797daa None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:34.237565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 21/40] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:28:34.243670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3b016-f6db-43c2-b4aa-b4e2dbaff785 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:34.244247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3b016-f6db-43c2-b4aa-b4e2dbaff785 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:34.244247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc3b016-f6db-43c2-b4aa-b4e2dbaff785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:34.263355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-bcc3b016-f6db-43c2-b4aa-b4e2dbaff785 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:34.264673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-bcc3b016-f6db-43c2-b4aa-b4e2dbaff785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:34.300705 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458964d-1f0b-4c65-b462-cf608505ab43 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:34.348585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 22/41] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:28:34.353029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9daf78a-501f-48af-a38d-aadb8919b5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:34.353835 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daf78a-501f-48af-a38d-aadb8919b5a7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:34.354250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daf78a-501f-48af-a38d-aadb8919b5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:34.354735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9daf78a-501f-48af-a38d-aadb8919b5a7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:34.356081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9daf78a-501f-48af-a38d-aadb8919b5a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:34.357741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9daf78a-501f-48af-a38d-aadb8919b5a7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:34.359433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 20/42] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:28:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:34.366099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3b016-f6db-43c2-b4aa-b4e2dbaff785 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:34.472615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc3b016-f6db-43c2-b4aa-b4e2dbaff785 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:34.489824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:34.506511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9daf78a-501f-48af-a38d-aadb8919b5a7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:34.509993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a92ad-c0c3-4906-8b5d-366e963f18c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:34.510666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a92ad-c0c3-4906-8b5d-366e963f18c5 None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:34.510947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a92ad-c0c3-4906-8b5d-366e963f18c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:34.511220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77a92ad-c0c3-4906-8b5d-366e963f18c5 None admin] PATH_INFO: `/v3/domains/default/users/fee2283327204c6f8ee811e2c566e379/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:34.513216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a92ad-c0c3-4906-8b5d-366e963f18c5 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=fee2283327204c6f8ee811e2c566e379, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:34.514668 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b77a92ad-c0c3-4906-8b5d-366e963f18c5 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:34.535221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 23/43] 10.222.0.39 () {58 vars in 1361 bytes} [Tue Sep 6 22:28:34 2022] PUT /identity/v3/domains/default/users/fee2283327204c6f8ee811e2c566e379/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:28:34.601999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:34.640289 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a92ad-c0c3-4906-8b5d-366e963f18c5 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:34.644693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9daf78a-501f-48af-a38d-aadb8919b5a7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:34.691499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a074428-2e06-4e7d-bb03-51e6f98b0c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:34.692198 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a074428-2e06-4e7d-bb03-51e6f98b0c34 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:34.692483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a074428-2e06-4e7d-bb03-51e6f98b0c34 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:34.693100 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a074428-2e06-4e7d-bb03-51e6f98b0c34 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:34.694364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a074428-2e06-4e7d-bb03-51e6f98b0c34 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:34.696092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a074428-2e06-4e7d-bb03-51e6f98b0c34 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:34.719534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 21/44] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Sep 6 22:28:34 2022] POST /identity/v3/roles => generated 223 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:28:34.759618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77a92ad-c0c3-4906-8b5d-366e963f18c5 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:34.797527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43049a03-0cf7-434c-b7eb-067aafff0499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:34.798182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049a03-0cf7-434c-b7eb-067aafff0499 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:34.798459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049a03-0cf7-434c-b7eb-067aafff0499 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:34.798728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049a03-0cf7-434c-b7eb-067aafff0499 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:34.799891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43049a03-0cf7-434c-b7eb-067aafff0499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:34.801396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43049a03-0cf7-434c-b7eb-067aafff0499 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:34.804792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 24/45] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:28:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:34.809993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a074428-2e06-4e7d-bb03-51e6f98b0c34 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:34.921042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a074428-2e06-4e7d-bb03-51e6f98b0c34 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:34.946693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:34.956372 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1f3926-7b23-4b87-a7f5-8b18b0314da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:34.957182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f3926-7b23-4b87-a7f5-8b18b0314da9 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:34.957628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f3926-7b23-4b87-a7f5-8b18b0314da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:34.958113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f3926-7b23-4b87-a7f5-8b18b0314da9 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:34.958763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f3926-7b23-4b87-a7f5-8b18b0314da9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:34.961142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1f3926-7b23-4b87-a7f5-8b18b0314da9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:34.989765 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 22/46] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:28:34 2022] POST /identity/v3/projects => generated 312 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:28:36.739084 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ed3e6-6c4c-44d4-84a1-264c20ad5099 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:36.739622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ed3e6-6c4c-44d4-84a1-264c20ad5099 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:36.739622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240ed3e6-6c4c-44d4-84a1-264c20ad5099 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:36.741422 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 25/47] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:36.746483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c9adf-394f-454b-8126-53896d17f330 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:36.746883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c9adf-394f-454b-8126-53896d17f330 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:36.747039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c9adf-394f-454b-8126-53896d17f330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:36.766581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-413c9adf-394f-454b-8126-53896d17f330 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:36.768099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-413c9adf-394f-454b-8126-53896d17f330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:36.833764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 23/48] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:37.107974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ed3e6-6c4c-44d4-84a1-264c20ad5099 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:37.156554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:37.182216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c919ca-8740-449e-9cdc-78fe6c3f634b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:37.182509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c919ca-8740-449e-9cdc-78fe6c3f634b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:37.182789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c919ca-8740-449e-9cdc-78fe6c3f634b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:37.184130 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 24/49] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:28:37.189104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda7d8e-8d3c-48f1-9b01-5235b6a0db0c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:37.189391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda7d8e-8d3c-48f1-9b01-5235b6a0db0c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:37.189672 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eda7d8e-8d3c-48f1-9b01-5235b6a0db0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:37.207856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4eda7d8e-8d3c-48f1-9b01-5235b6a0db0c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:37.209115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4eda7d8e-8d3c-48f1-9b01-5235b6a0db0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:37.211887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240ed3e6-6c4c-44d4-84a1-264c20ad5099 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:37.244767 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253139d-6ddd-4709-85a5-93ae1308dbf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:37.245405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253139d-6ddd-4709-85a5-93ae1308dbf9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:37.245685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253139d-6ddd-4709-85a5-93ae1308dbf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:37.245954 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253139d-6ddd-4709-85a5-93ae1308dbf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:37.255895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253139d-6ddd-4709-85a5-93ae1308dbf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-5253139d-6ddd-4709-85a5-93ae1308dbf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:37.262130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:37.262966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:28:37.263744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 26/50] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:28:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:37.268929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253139d-6ddd-4709-85a5-93ae1308dbf9 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:37.277127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 25/51] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:28:37.364297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253139d-6ddd-4709-85a5-93ae1308dbf9 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:37.366984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:37.398211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ed256-9b93-4a9c-881c-ee9d81b0737c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:37.398815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ed256-9b93-4a9c-881c-ee9d81b0737c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:37.399094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ed256-9b93-4a9c-881c-ee9d81b0737c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:37.399369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ed256-9b93-4a9c-881c-ee9d81b0737c None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:37.400029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ed256-9b93-4a9c-881c-ee9d81b0737c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:37.400333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ed256-9b93-4a9c-881c-ee9d81b0737c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:37.401745 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ed256-9b93-4a9c-881c-ee9d81b0737c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:37.409902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 27/52] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:37.415599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda7d8e-8d3c-48f1-9b01-5235b6a0db0c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:37.454535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:37.521050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eda7d8e-8d3c-48f1-9b01-5235b6a0db0c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:37.549149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ed256-9b93-4a9c-881c-ee9d81b0737c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:37.555037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec62133-bf37-46f2-9582-ee50dc815adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:37.555676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec62133-bf37-46f2-9582-ee50dc815adf None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:37.555951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec62133-bf37-46f2-9582-ee50dc815adf None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:37.556219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec62133-bf37-46f2-9582-ee50dc815adf None admin] PATH_INFO: `/v3/users/fee2283327204c6f8ee811e2c566e379` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:37.557359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec62133-bf37-46f2-9582-ee50dc815adf None admin] RBAC: Authorizing `identity:get_user(user_id=fee2283327204c6f8ee811e2c566e379)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:37.558805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec62133-bf37-46f2-9582-ee50dc815adf None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:37.560364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 26/53] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:37 2022] GET /identity/v3/users/fee2283327204c6f8ee811e2c566e379 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:37.565576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec62133-bf37-46f2-9582-ee50dc815adf None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:37.650663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ed256-9b93-4a9c-881c-ee9d81b0737c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:37.663485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec62133-bf37-46f2-9582-ee50dc815adf None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:37.670667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:37.686007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe902ae-a8af-4e62-951c-1860897f1b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:37.686827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe902ae-a8af-4e62-951c-1860897f1b2c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:37.687107 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe902ae-a8af-4e62-951c-1860897f1b2c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:37.687375 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe902ae-a8af-4e62-951c-1860897f1b2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:37.688474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe902ae-a8af-4e62-951c-1860897f1b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:37.689941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe902ae-a8af-4e62-951c-1860897f1b2c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:37.691489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 28/54] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:28:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:37.697702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe902ae-a8af-4e62-951c-1860897f1b2c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:37.698772 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd965a5-f69a-4aa2-a8a6-c910c8ec1bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:37.699404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd965a5-f69a-4aa2-a8a6-c910c8ec1bcc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:37.699681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd965a5-f69a-4aa2-a8a6-c910c8ec1bcc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:37.699958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd965a5-f69a-4aa2-a8a6-c910c8ec1bcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:37.701105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd965a5-f69a-4aa2-a8a6-c910c8ec1bcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:37.702543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd965a5-f69a-4aa2-a8a6-c910c8ec1bcc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:37.704084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 27/55] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:28:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:37.709651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd965a5-f69a-4aa2-a8a6-c910c8ec1bcc None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:37.734278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:37.800874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe902ae-a8af-4e62-951c-1860897f1b2c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:37.810544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd965a5-f69a-4aa2-a8a6-c910c8ec1bcc None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:37.847238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b0783-89e4-44c0-bdd5-2f5d955fe036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:37.847877 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b0783-89e4-44c0-bdd5-2f5d955fe036 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:37.848150 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b0783-89e4-44c0-bdd5-2f5d955fe036 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:37.848432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b0783-89e4-44c0-bdd5-2f5d955fe036 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:37.849213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b0783-89e4-44c0-bdd5-2f5d955fe036 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.domain.id=default, user.id=fee2283327204c6f8ee811e2c566e379) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:37.849527 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b0783-89e4-44c0-bdd5-2f5d955fe036 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:37.851835 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b0783-89e4-44c0-bdd5-2f5d955fe036 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:37.858464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 28/56] 10.222.0.39 () {56 vars in 1322 bytes} [Tue Sep 6 22:28:37 2022] GET /identity/v3/role_assignments?user.id=fee2283327204c6f8ee811e2c566e379&scope.domain.id=default&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:37.948788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af6ae40-638a-4555-9d16-6e5393fded81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:37.949331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6ae40-638a-4555-9d16-6e5393fded81 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:37.949612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6ae40-638a-4555-9d16-6e5393fded81 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:37.949884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af6ae40-638a-4555-9d16-6e5393fded81 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:37.950553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6ae40-638a-4555-9d16-6e5393fded81 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:37.952023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af6ae40-638a-4555-9d16-6e5393fded81 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:37.985839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 29/57] 10.222.0.39 () {60 vars in 1140 bytes} [Tue Sep 6 22:28:37 2022] POST /identity/v3/projects => generated 298 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:28:40.150117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc362eef-a1d7-4cab-ae15-265be02c36ff None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:40.150646 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc362eef-a1d7-4cab-ae15-265be02c36ff None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:40.150646 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc362eef-a1d7-4cab-ae15-265be02c36ff None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:40.151988 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 29/58] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:40.157106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df2c25-6d52-43fa-8bbe-d11206f87461 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:40.157510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df2c25-6d52-43fa-8bbe-d11206f87461 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:40.157649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9df2c25-6d52-43fa-8bbe-d11206f87461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:40.176052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e9df2c25-6d52-43fa-8bbe-d11206f87461 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:40.177248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e9df2c25-6d52-43fa-8bbe-d11206f87461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:40.240115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 30/59] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:40.513495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc362eef-a1d7-4cab-ae15-265be02c36ff None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:40.556911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:40.617615 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc362eef-a1d7-4cab-ae15-265be02c36ff None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:40.651889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e835298-0cd7-4346-8c12-53504d125090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:40.652544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e835298-0cd7-4346-8c12-53504d125090 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:40.652852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e835298-0cd7-4346-8c12-53504d125090 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:40.653128 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e835298-0cd7-4346-8c12-53504d125090 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:40.654240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e835298-0cd7-4346-8c12-53504d125090 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:40.655665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e835298-0cd7-4346-8c12-53504d125090 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:40.657238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 30/60] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:28:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:40.662954 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df2c25-6d52-43fa-8bbe-d11206f87461 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:40.680488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:40.760084 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9df2c25-6d52-43fa-8bbe-d11206f87461 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:40.792957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b5145f-0b06-4c0d-a436-6cb210f1f733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:40.793482 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5145f-0b06-4c0d-a436-6cb210f1f733 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:40.793758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5145f-0b06-4c0d-a436-6cb210f1f733 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:40.794027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b5145f-0b06-4c0d-a436-6cb210f1f733 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:40.794673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5145f-0b06-4c0d-a436-6cb210f1f733 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:40.796083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b5145f-0b06-4c0d-a436-6cb210f1f733 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:40.822737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 31/61] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:28:40 2022] POST /identity/v3/users => generated 275 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:28:44.001654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d292-f81f-4d93-9661-b8f5b839c9a0 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:44.002680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d292-f81f-4d93-9661-b8f5b839c9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:44.003250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc2d292-f81f-4d93-9661-b8f5b839c9a0 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:44.004809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 31/62] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:44.010940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d371b6f-73d6-4f69-ba75-e6fc9e1343d5 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:44.011763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d371b6f-73d6-4f69-ba75-e6fc9e1343d5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:44.012257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d371b6f-73d6-4f69-ba75-e6fc9e1343d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:44.041836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-9d371b6f-73d6-4f69-ba75-e6fc9e1343d5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:44.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9d371b6f-73d6-4f69-ba75-e6fc9e1343d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:44.121252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 32/63] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:44.428684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2d292-f81f-4d93-9661-b8f5b839c9a0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:44.586267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14352b79-e7fd-471c-bb34-0f4a30dc4783 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:44.586928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14352b79-e7fd-471c-bb34-0f4a30dc4783 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:44.587519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14352b79-e7fd-471c-bb34-0f4a30dc4783 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:44.587915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc2d292-f81f-4d93-9661-b8f5b839c9a0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:44.589397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 33/64] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:44.595425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247da76-0e15-4ff7-a4a2-9a7e2d618870 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:44.596056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247da76-0e15-4ff7-a4a2-9a7e2d618870 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:44.596529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e247da76-0e15-4ff7-a4a2-9a7e2d618870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:44.600517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:44.614554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:44.637081 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e247da76-0e15-4ff7-a4a2-9a7e2d618870 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:44.639603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e247da76-0e15-4ff7-a4a2-9a7e2d618870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:44.666352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714135d-bdc5-4558-9031-b598f6eca492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:44.667370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714135d-bdc5-4558-9031-b598f6eca492 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:44.667855 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714135d-bdc5-4558-9031-b598f6eca492 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:44.668331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a714135d-bdc5-4558-9031-b598f6eca492 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:44.692866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a714135d-bdc5-4558-9031-b598f6eca492 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a714135d-bdc5-4558-9031-b598f6eca492 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:44.699199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:44.700171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:44.700931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:28:44.705702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 32/65] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:44.712813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714135d-bdc5-4558-9031-b598f6eca492 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:44.800219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 34/66] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:44.806471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1a3a6-1295-4516-be50-d359cf92a3bb None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:44.806757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1a3a6-1295-4516-be50-d359cf92a3bb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:44.807042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1a3a6-1295-4516-be50-d359cf92a3bb None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:44.808833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 35/67] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:44 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:44.814045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e9963-b72c-40b2-b32b-f8b8c112a65a None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:44.814576 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e9963-b72c-40b2-b32b-f8b8c112a65a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:44.815387 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28e9963-b72c-40b2-b32b-f8b8c112a65a None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:44.818022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 36/68] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:44.819351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9b699-a345-47c3-b224-7ac5436f63b3 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:44.819801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9b699-a345-47c3-b224-7ac5436f63b3 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:44.820244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9b699-a345-47c3-b224-7ac5436f63b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:44.842478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-06e9b699-a345-47c3-b224-7ac5436f63b3 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:44.844037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-06e9b699-a345-47c3-b224-7ac5436f63b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:44.875682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a714135d-bdc5-4558-9031-b598f6eca492 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:44.914569 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a4ee3-d83c-4a02-a560-af088efe5e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:44.915442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a4ee3-d83c-4a02-a560-af088efe5e05 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:44.915901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a4ee3-d83c-4a02-a560-af088efe5e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:44.916406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a4ee3-d83c-4a02-a560-af088efe5e05 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:44.917314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a4ee3-d83c-4a02-a560-af088efe5e05 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:44.917802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a4ee3-d83c-4a02-a560-af088efe5e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:44.919425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a4ee3-d83c-4a02-a560-af088efe5e05 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:44.928785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 37/69] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:44.928785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc12e02-0813-49d1-9e87-3fefc6ae268a None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:44.928785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc12e02-0813-49d1-9e87-3fefc6ae268a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:44.928785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc12e02-0813-49d1-9e87-3fefc6ae268a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:44.945196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 33/70] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:28:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:44.946719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-cdc12e02-0813-49d1-9e87-3fefc6ae268a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:44.947933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-cdc12e02-0813-49d1-9e87-3fefc6ae268a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:44.951790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a4ee3-d83c-4a02-a560-af088efe5e05 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:45.009546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:45.021736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 38/71] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:45.023774 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:45.036547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7944f66-18d8-4745-bb84-7af6af91506e None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:45.036867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7944f66-18d8-4745-bb84-7af6af91506e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:45.037807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7944f66-18d8-4745-bb84-7af6af91506e None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:45.039393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 39/72] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:28:45.044264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1ac0d-7f7b-4db8-88ff-7c412a89c0be None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:45.044551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1ac0d-7f7b-4db8-88ff-7c412a89c0be None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:45.044863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1ac0d-7f7b-4db8-88ff-7c412a89c0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:45.063860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-0bc1ac0d-7f7b-4db8-88ff-7c412a89c0be None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:45.065122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-0bc1ac0d-7f7b-4db8-88ff-7c412a89c0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:45.137971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 40/73] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:28:45.174298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a4ee3-d83c-4a02-a560-af088efe5e05 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:45.175477 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1ac0d-7f7b-4db8-88ff-7c412a89c0be None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:45.228467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fd604-188c-4964-aed3-aec93391e67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:45.229050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fd604-188c-4964-aed3-aec93391e67a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:45.229328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fd604-188c-4964-aed3-aec93391e67a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:45.230052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8fd604-188c-4964-aed3-aec93391e67a None admin] PATH_INFO: `/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:45.245960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fd604-188c-4964-aed3-aec93391e67a None admin] RBAC: Authorizing `identity:get_user(user_id=0cac9a0d6d824acfbc13df63c8e5dff8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:45.247631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8fd604-188c-4964-aed3-aec93391e67a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:45.249439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 34/74] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:44 2022] GET /identity/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8 => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:45.256774 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fd604-188c-4964-aed3-aec93391e67a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:45.339208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1ac0d-7f7b-4db8-88ff-7c412a89c0be None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:45.347990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:45.363316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:45.363880 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8fd604-188c-4964-aed3-aec93391e67a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:45.376309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f9063-327e-49da-8ab0-c0a5e1aeef56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:45.376981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f9063-327e-49da-8ab0-c0a5e1aeef56 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:45.377285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f9063-327e-49da-8ab0-c0a5e1aeef56 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:45.377562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f9063-327e-49da-8ab0-c0a5e1aeef56 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:45.389492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f9063-327e-49da-8ab0-c0a5e1aeef56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-cb3f9063-327e-49da-8ab0-c0a5e1aeef56 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:45.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:45.395178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:28:45.395909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 41/75] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:28:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:28:45.397225 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f9063-327e-49da-8ab0-c0a5e1aeef56 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:45.403809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918cc03-dbc0-4fde-a2db-8cbb553c16d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:45.404747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918cc03-dbc0-4fde-a2db-8cbb553c16d4 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:45.405114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918cc03-dbc0-4fde-a2db-8cbb553c16d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:45.405382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918cc03-dbc0-4fde-a2db-8cbb553c16d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:45.416687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918cc03-dbc0-4fde-a2db-8cbb553c16d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-6918cc03-dbc0-4fde-a2db-8cbb553c16d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:45.421901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:45.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:28:45.423513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 35/76] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:45.424144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918cc03-dbc0-4fde-a2db-8cbb553c16d4 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:45.527626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f9063-327e-49da-8ab0-c0a5e1aeef56 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:45.533581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918cc03-dbc0-4fde-a2db-8cbb553c16d4 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:45.561036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edb647-1b2f-4c89-952f-0a3c5a363dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:45.561634 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edb647-1b2f-4c89-952f-0a3c5a363dca None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:45.561915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edb647-1b2f-4c89-952f-0a3c5a363dca None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:45.562179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edb647-1b2f-4c89-952f-0a3c5a363dca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:45.567366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada16166-dacb-4bfa-bb9f-c377d7cdbe7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:45.567992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada16166-dacb-4bfa-bb9f-c377d7cdbe7c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:45.568432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada16166-dacb-4bfa-bb9f-c377d7cdbe7c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:45.568432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada16166-dacb-4bfa-bb9f-c377d7cdbe7c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:45.582194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada16166-dacb-4bfa-bb9f-c377d7cdbe7c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-ada16166-dacb-4bfa-bb9f-c377d7cdbe7c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:45.588142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:45.589097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:28:45.589915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 36/77] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:28:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:45.590532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada16166-dacb-4bfa-bb9f-c377d7cdbe7c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:45.600527 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:45.616325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:45.635713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edb647-1b2f-4c89-952f-0a3c5a363dca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:45.637224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edb647-1b2f-4c89-952f-0a3c5a363dca None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:45.691491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada16166-dacb-4bfa-bb9f-c377d7cdbe7c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:45.726922 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6318e29-901c-483c-aed2-ca7b25f7a34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:45.727528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6318e29-901c-483c-aed2-ca7b25f7a34b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:45.727810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6318e29-901c-483c-aed2-ca7b25f7a34b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:45.728091 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6318e29-901c-483c-aed2-ca7b25f7a34b None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:45.728846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6318e29-901c-483c-aed2-ca7b25f7a34b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:45.729091 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6318e29-901c-483c-aed2-ca7b25f7a34b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:45.730576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6318e29-901c-483c-aed2-ca7b25f7a34b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:45.737344 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 42/78] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:28:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:28:45.738185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edb647-1b2f-4c89-952f-0a3c5a363dca None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:45.738405 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 37/79] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:45.741647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6318e29-901c-483c-aed2-ca7b25f7a34b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:45.826867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:45.827812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:45.848418 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edb647-1b2f-4c89-952f-0a3c5a363dca None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:45.861847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6318e29-901c-483c-aed2-ca7b25f7a34b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:45.882584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3bc66-ce83-46ae-a800-96c237586372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:45.883189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3bc66-ce83-46ae-a800-96c237586372 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:45.883467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3bc66-ce83-46ae-a800-96c237586372 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:45.883739 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b3bc66-ce83-46ae-a800-96c237586372 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:45.884398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3bc66-ce83-46ae-a800-96c237586372 None admin] RBAC: Adding query filter params (name=member) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:45.884705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3bc66-ce83-46ae-a800-96c237586372 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:45.886106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b3bc66-ce83-46ae-a800-96c237586372 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:45.893903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 43/80] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:28:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:45.895301 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527501b-0337-469d-b6bf-67f47dc3d625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:45.895963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527501b-0337-469d-b6bf-67f47dc3d625 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:45.896274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527501b-0337-469d-b6bf-67f47dc3d625 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:45.897167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527501b-0337-469d-b6bf-67f47dc3d625 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:45.898256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3bc66-ce83-46ae-a800-96c237586372 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:45.898372 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527501b-0337-469d-b6bf-67f47dc3d625 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:45.899904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527501b-0337-469d-b6bf-67f47dc3d625 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:45.901642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 38/81] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:28:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:45.902333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527501b-0337-469d-b6bf-67f47dc3d625 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:45.995485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b3bc66-ce83-46ae-a800-96c237586372 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:45.998788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527501b-0337-469d-b6bf-67f47dc3d625 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:46.029276 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe16a3-1e56-4f5f-bd25-3929180f5bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:46.029875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe16a3-1e56-4f5f-bd25-3929180f5bb5 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:46.030153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe16a3-1e56-4f5f-bd25-3929180f5bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:46.030425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fe16a3-1e56-4f5f-bd25-3929180f5bb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:46.031077 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe16a3-1e56-4f5f-bd25-3929180f5bb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:46.031375 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe16a3-1e56-4f5f-bd25-3929180f5bb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:46.032017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fd00c-ea6e-4054-b637-affe07c3e801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:46.032639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fd00c-ea6e-4054-b637-affe07c3e801 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:46.032726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fe16a3-1e56-4f5f-bd25-3929180f5bb5 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:46.032950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fd00c-ea6e-4054-b637-affe07c3e801 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:46.033224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292fd00c-ea6e-4054-b637-affe07c3e801 None admin] PATH_INFO: `/v3/users/fee2283327204c6f8ee811e2c566e379` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:46.034355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fd00c-ea6e-4054-b637-affe07c3e801 None admin] RBAC: Authorizing `identity:get_user(user_id=fee2283327204c6f8ee811e2c566e379)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:46.035778 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292fd00c-ea6e-4054-b637-affe07c3e801 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:46.037893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 39/82] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:45 2022] GET /identity/v3/users/fee2283327204c6f8ee811e2c566e379 => generated 247 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:46.038764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fd00c-ea6e-4054-b637-affe07c3e801 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:46.044804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 44/83] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:28:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:46.044804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe16a3-1e56-4f5f-bd25-3929180f5bb5 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:46.049211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:46.051851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:46.145174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fe16a3-1e56-4f5f-bd25-3929180f5bb5 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:46.148928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292fd00c-ea6e-4054-b637-affe07c3e801 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:46.182123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de180a97-3b57-456d-9340-a08d3b04eb96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:46.182753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de180a97-3b57-456d-9340-a08d3b04eb96 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:46.183035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de180a97-3b57-456d-9340-a08d3b04eb96 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:46.183311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de180a97-3b57-456d-9340-a08d3b04eb96 None admin] PATH_INFO: `/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:46.184446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de180a97-3b57-456d-9340-a08d3b04eb96 None admin] RBAC: Authorizing `identity:get_user(user_id=0cac9a0d6d824acfbc13df63c8e5dff8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:46.184855 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2f2c5-d0f5-4a1a-a9ec-501545e2455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:46.185482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2f2c5-d0f5-4a1a-a9ec-501545e2455c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:46.186214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2f2c5-d0f5-4a1a-a9ec-501545e2455c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:46.186313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de180a97-3b57-456d-9340-a08d3b04eb96 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:46.186397 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2f2c5-d0f5-4a1a-a9ec-501545e2455c None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:46.187197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2f2c5-d0f5-4a1a-a9ec-501545e2455c None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:46.187771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 45/84] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:46 2022] GET /identity/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:46.188351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de180a97-3b57-456d-9340-a08d3b04eb96 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:46.188710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b2f2c5-d0f5-4a1a-a9ec-501545e2455c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:46.190489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 40/85] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:46 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:46.191108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2f2c5-d0f5-4a1a-a9ec-501545e2455c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:46.262236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:46.265006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:46.289778 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de180a97-3b57-456d-9340-a08d3b04eb96 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:46.296358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2f2c5-d0f5-4a1a-a9ec-501545e2455c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:46.322915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e9730-9eec-43ea-bf56-bf80a01e3867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:46.323526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e9730-9eec-43ea-bf56-bf80a01e3867 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:46.323802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e9730-9eec-43ea-bf56-bf80a01e3867 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:46.324076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624e9730-9eec-43ea-bf56-bf80a01e3867 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:46.324720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e9730-9eec-43ea-bf56-bf80a01e3867 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:46.326192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624e9730-9eec-43ea-bf56-bf80a01e3867 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:46.331009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796574e4-49b2-4134-99e5-ed319e6a9e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:46.331615 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796574e4-49b2-4134-99e5-ed319e6a9e8c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:46.331899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796574e4-49b2-4134-99e5-ed319e6a9e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:46.332173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796574e4-49b2-4134-99e5-ed319e6a9e8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:46.347327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 46/86] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:28:46 2022] POST /identity/v3/users => generated 289 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:28:46.348129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e9730-9eec-43ea-bf56-bf80a01e3867 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:46.414751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796574e4-49b2-4134-99e5-ed319e6a9e8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:46.416919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796574e4-49b2-4134-99e5-ed319e6a9e8c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:46.471796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624e9730-9eec-43ea-bf56-bf80a01e3867 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:46.502526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:46.503809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:46.513498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75598cf2-1311-4312-9977-c10e07d7da1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:46.513866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75598cf2-1311-4312-9977-c10e07d7da1e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:46.514663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75598cf2-1311-4312-9977-c10e07d7da1e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:46.514938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75598cf2-1311-4312-9977-c10e07d7da1e None admin] PATH_INFO: `/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:46.516216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75598cf2-1311-4312-9977-c10e07d7da1e None admin] RBAC: Authorizing `identity:get_user(user_id=0cac9a0d6d824acfbc13df63c8e5dff8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:46.517662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75598cf2-1311-4312-9977-c10e07d7da1e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:46.519498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 47/87] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:46 2022] GET /identity/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:46.520070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75598cf2-1311-4312-9977-c10e07d7da1e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:46.550501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 41/88] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:28:46.551651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796574e4-49b2-4134-99e5-ed319e6a9e8c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:46.655459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75598cf2-1311-4312-9977-c10e07d7da1e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:46.682856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796574e4-49b2-4134-99e5-ed319e6a9e8c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:46.702544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2a0cc-71c9-4ef8-a011-4342445330ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:46.703163 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a0cc-71c9-4ef8-a011-4342445330ad None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:46.703446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a0cc-71c9-4ef8-a011-4342445330ad None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:46.703720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a0cc-71c9-4ef8-a011-4342445330ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:46.721122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb92ed7-d23b-4929-bd03-4bb9c0ff21fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:46.721739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb92ed7-d23b-4929-bd03-4bb9c0ff21fc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:46.722020 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb92ed7-d23b-4929-bd03-4bb9c0ff21fc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:46.722292 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb92ed7-d23b-4929-bd03-4bb9c0ff21fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:46.723024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb92ed7-d23b-4929-bd03-4bb9c0ff21fc None admin] RBAC: Adding query filter params (role.id=aac9ca64da09463ba9b3966c7188d7ee, scope.project.id=a6cc40a4154043ada48161d66116e3c8, user.id=0cac9a0d6d824acfbc13df63c8e5dff8) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:46.723332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb92ed7-d23b-4929-bd03-4bb9c0ff21fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:46.727912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb92ed7-d23b-4929-bd03-4bb9c0ff21fc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:46.736869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 42/89] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:28:46 2022] GET /identity/v3/role_assignments?user.id=0cac9a0d6d824acfbc13df63c8e5dff8&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:46.736869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb92ed7-d23b-4929-bd03-4bb9c0ff21fc None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:46.779638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:46.850408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:46.859464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2a0cc-71c9-4ef8-a011-4342445330ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:46.868491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2a0cc-71c9-4ef8-a011-4342445330ad None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:46.912860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb92ed7-d23b-4929-bd03-4bb9c0ff21fc None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:46.986323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524751c-13fc-46c5-b439-4d8922dd7409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:46.987289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524751c-13fc-46c5-b439-4d8922dd7409 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:46.987998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524751c-13fc-46c5-b439-4d8922dd7409 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:46.989524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0524751c-13fc-46c5-b439-4d8922dd7409 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:47.064251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 48/90] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 544 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:28:47.070356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2a0cc-71c9-4ef8-a011-4342445330ad None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:47.087713 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524751c-13fc-46c5-b439-4d8922dd7409 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:47.089223 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0524751c-13fc-46c5-b439-4d8922dd7409 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:47.179271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:47.185492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2a0cc-71c9-4ef8-a011-4342445330ad None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:47.186967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:47.228526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636573c-db55-417f-9a1c-3a21a25e12c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:47.229934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636573c-db55-417f-9a1c-3a21a25e12c4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:47.230440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636573c-db55-417f-9a1c-3a21a25e12c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:47.230920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a636573c-db55-417f-9a1c-3a21a25e12c4 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:47.232268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636573c-db55-417f-9a1c-3a21a25e12c4 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:47.234467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a636573c-db55-417f-9a1c-3a21a25e12c4 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:47.236321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 49/91] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:47 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:47.237463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636573c-db55-417f-9a1c-3a21a25e12c4 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:47.350003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 43/92] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:28:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 616 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:28:47.351282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524751c-13fc-46c5-b439-4d8922dd7409 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:47.363652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a636573c-db55-417f-9a1c-3a21a25e12c4 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:47.461351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0524751c-13fc-46c5-b439-4d8922dd7409 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:47.462918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8404951-94ea-4c66-b0fc-01da6253ef8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:47.463594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8404951-94ea-4c66-b0fc-01da6253ef8e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:47.463840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8404951-94ea-4c66-b0fc-01da6253ef8e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:47.464116 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8404951-94ea-4c66-b0fc-01da6253ef8e None admin] PATH_INFO: `/v3/projects/929f910a2e0d4acc995c08172ce891f3` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:47.469466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8404951-94ea-4c66-b0fc-01da6253ef8e None admin] RBAC: Authorizing `identity:get_project(project_id=929f910a2e0d4acc995c08172ce891f3)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:47.471071 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8404951-94ea-4c66-b0fc-01da6253ef8e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:47.472575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 50/93] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:47 2022] GET /identity/v3/projects/929f910a2e0d4acc995c08172ce891f3 => generated 312 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:47.476896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8404951-94ea-4c66-b0fc-01da6253ef8e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:47.508300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a797d-3a84-4ba9-a4ef-5ce3f2779051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:47.508978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a797d-3a84-4ba9-a4ef-5ce3f2779051 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:47.509328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a797d-3a84-4ba9-a4ef-5ce3f2779051 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:47.509671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1a797d-3a84-4ba9-a4ef-5ce3f2779051 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:47.510481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a797d-3a84-4ba9-a4ef-5ce3f2779051 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=a6cc40a4154043ada48161d66116e3c8, user.id=fee2283327204c6f8ee811e2c566e379) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:47.510854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a797d-3a84-4ba9-a4ef-5ce3f2779051 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:47.512416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1a797d-3a84-4ba9-a4ef-5ce3f2779051 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:47.516073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:47.520108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 44/94] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:28:47 2022] GET /identity/v3/role_assignments?user.id=fee2283327204c6f8ee811e2c566e379&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:47.520868 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a797d-3a84-4ba9-a4ef-5ce3f2779051 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:47.591425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:47.642000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1a797d-3a84-4ba9-a4ef-5ce3f2779051 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:47.680136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd03ae9-431b-4785-b44b-7e513812e540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:47.680776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03ae9-431b-4785-b44b-7e513812e540 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:47.681062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03ae9-431b-4785-b44b-7e513812e540 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:47.681331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd03ae9-431b-4785-b44b-7e513812e540 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:47.682066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd03ae9-431b-4785-b44b-7e513812e540 None admin] RBAC: Adding query filter params (role.id=aac9ca64da09463ba9b3966c7188d7ee, scope.project.id=929f910a2e0d4acc995c08172ce891f3, user.id=0cac9a0d6d824acfbc13df63c8e5dff8) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:47.682370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd03ae9-431b-4785-b44b-7e513812e540 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:47.683921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd03ae9-431b-4785-b44b-7e513812e540 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:47.690609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 45/95] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:28:47 2022] GET /identity/v3/role_assignments?user.id=0cac9a0d6d824acfbc13df63c8e5dff8&scope.project.id=929f910a2e0d4acc995c08172ce891f3&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:47.737066 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8404951-94ea-4c66-b0fc-01da6253ef8e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:47.794110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa935ad-3e04-4887-81f0-f67b557274a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:47.794681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa935ad-3e04-4887-81f0-f67b557274a9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:47.794962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa935ad-3e04-4887-81f0-f67b557274a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:47.795233 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa935ad-3e04-4887-81f0-f67b557274a9 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:47.796522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa935ad-3e04-4887-81f0-f67b557274a9 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:47.798022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa935ad-3e04-4887-81f0-f67b557274a9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:47.799941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 51/96] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:47 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:47.806736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd03ae9-431b-4785-b44b-7e513812e540 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:47.887057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:47.930355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd03ae9-431b-4785-b44b-7e513812e540 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:47.967199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cf9080-9fc8-4df4-a17d-b057d17c79a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:47.967822 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf9080-9fc8-4df4-a17d-b057d17c79a7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:47.968100 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf9080-9fc8-4df4-a17d-b057d17c79a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:47.968371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cf9080-9fc8-4df4-a17d-b057d17c79a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:47.969222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf9080-9fc8-4df4-a17d-b057d17c79a7 None admin] RBAC: Adding query filter params (role.id=952084593c184a78a952e1797a0b6c6b, scope.project.id=a6cc40a4154043ada48161d66116e3c8, user.id=0cac9a0d6d824acfbc13df63c8e5dff8) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:47.969530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf9080-9fc8-4df4-a17d-b057d17c79a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:47.971093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cf9080-9fc8-4df4-a17d-b057d17c79a7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:47.977852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 46/97] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:28:47 2022] GET /identity/v3/role_assignments?user.id=0cac9a0d6d824acfbc13df63c8e5dff8&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=952084593c184a78a952e1797a0b6c6b => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:49.265842 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a991a-ec09-4f11-8b46-9d1e41f18273 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:49.266796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a991a-ec09-4f11-8b46-9d1e41f18273 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:49.267322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8a991a-ec09-4f11-8b46-9d1e41f18273 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:49.268894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 52/98] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:49.274936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9321f-67b1-45c3-a00f-499a394c4e40 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:49.275357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9321f-67b1-45c3-a00f-499a394c4e40 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:49.275526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc9321f-67b1-45c3-a00f-499a394c4e40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:49.301764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-cfc9321f-67b1-45c3-a00f-499a394c4e40 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:49.303713 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cfc9321f-67b1-45c3-a00f-499a394c4e40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:49.388903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 47/99] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:49.514483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8bdde-a82b-4731-8002-09fabd5669bd None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:49.515170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8bdde-a82b-4731-8002-09fabd5669bd None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:49.515672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b8bdde-a82b-4731-8002-09fabd5669bd None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:49.517555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 53/100] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:49.522510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fb467-1361-4d12-9225-7359ac037f48 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:49.522793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fb467-1361-4d12-9225-7359ac037f48 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:49.523063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58fb467-1361-4d12-9225-7359ac037f48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:49.556784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c58fb467-1361-4d12-9225-7359ac037f48 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:49.556784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c58fb467-1361-4d12-9225-7359ac037f48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:49.642000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:49.649880 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 48/101] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:49.880098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8bdde-a82b-4731-8002-09fabd5669bd None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:50.000301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b8bdde-a82b-4731-8002-09fabd5669bd None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:50.002175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:50.037729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb28a21-d2fa-4a68-b6f1-0244eeb191b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:50.038683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb28a21-d2fa-4a68-b6f1-0244eeb191b3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.039181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb28a21-d2fa-4a68-b6f1-0244eeb191b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.039642 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb28a21-d2fa-4a68-b6f1-0244eeb191b3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.055311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb28a21-d2fa-4a68-b6f1-0244eeb191b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-0eb28a21-d2fa-4a68-b6f1-0244eeb191b3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:50.060673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:50.061638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:28:50.062376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:28:50.064649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 54/102] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:49 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:50.071575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fb467-1361-4d12-9225-7359ac037f48 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:50.164019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb28a21-d2fa-4a68-b6f1-0244eeb191b3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:50.235289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58fb467-1361-4d12-9225-7359ac037f48 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:50.317518 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9668d07f-7355-4cb5-b071-3ad18277ae32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:50.318752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668d07f-7355-4cb5-b071-3ad18277ae32 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.318943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668d07f-7355-4cb5-b071-3ad18277ae32 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.319373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9668d07f-7355-4cb5-b071-3ad18277ae32 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.320203 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668d07f-7355-4cb5-b071-3ad18277ae32 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:50.320650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668d07f-7355-4cb5-b071-3ad18277ae32 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:50.322219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9668d07f-7355-4cb5-b071-3ad18277ae32 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:50.331148 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 49/103] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:28:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:50.332632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f3015-a987-434d-9b91-9833f864b3e5 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.333154 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f3015-a987-434d-9b91-9833f864b3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.333694 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4f3015-a987-434d-9b91-9833f864b3e5 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.335517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 50/104] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:50.340635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f3015-a987-434d-9b91-9833f864b3e5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:50.344538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb28a21-d2fa-4a68-b6f1-0244eeb191b3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:50.403801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:50.404421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91e6a6-351e-4e94-95b1-1b6071579895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:50.408310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91e6a6-351e-4e94-95b1-1b6071579895 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.408717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91e6a6-351e-4e94-95b1-1b6071579895 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.409128 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91e6a6-351e-4e94-95b1-1b6071579895 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.410709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91e6a6-351e-4e94-95b1-1b6071579895 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:50.414008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91e6a6-351e-4e94-95b1-1b6071579895 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:50.419447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 55/105] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:28:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:50.420712 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed1c3d-e30b-4f4a-b6d5-b051bad20008 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.422610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed1c3d-e30b-4f4a-b6d5-b051bad20008 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.423941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ed1c3d-e30b-4f4a-b6d5-b051bad20008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.453387 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-81ed1c3d-e30b-4f4a-b6d5-b051bad20008 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:50.454819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-81ed1c3d-e30b-4f4a-b6d5-b051bad20008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:50.479104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4f3015-a987-434d-9b91-9833f864b3e5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:50.481999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:50.525976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fef32f7-d92a-4b7b-8c51-d367681cf403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:50.526967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef32f7-d92a-4b7b-8c51-d367681cf403 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.527255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef32f7-d92a-4b7b-8c51-d367681cf403 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.527527 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fef32f7-d92a-4b7b-8c51-d367681cf403 None admin] PATH_INFO: `/v3/users/2ab0da9f2e6f43dc9337b9fc7bc13c2a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.541593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 56/106] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:50.542504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed1c3d-e30b-4f4a-b6d5-b051bad20008 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:50.558831 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef32f7-d92a-4b7b-8c51-d367681cf403 None admin] RBAC: Authorizing `identity:get_user(user_id=2ab0da9f2e6f43dc9337b9fc7bc13c2a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:50.560313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fef32f7-d92a-4b7b-8c51-d367681cf403 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:50.564075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 51/107] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:50 2022] GET /identity/v3/users/2ab0da9f2e6f43dc9337b9fc7bc13c2a => generated 289 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:50.565318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe128c-7ca8-4850-877e-9cc067e674ec None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.565604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe128c-7ca8-4850-877e-9cc067e674ec None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.565883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe128c-7ca8-4850-877e-9cc067e674ec None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.568802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 52/108] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:50.569565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe128c-7ca8-4850-877e-9cc067e674ec None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:50.743254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe128c-7ca8-4850-877e-9cc067e674ec None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:50.757964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:50.762250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ed1c3d-e30b-4f4a-b6d5-b051bad20008 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:50.798976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf88990-b45a-49e3-8ee6-378e51ae5adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:50.799863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88990-b45a-49e3-8ee6-378e51ae5adf None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.800332 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88990-b45a-49e3-8ee6-378e51ae5adf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.800736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf88990-b45a-49e3-8ee6-378e51ae5adf None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.801819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf88990-b45a-49e3-8ee6-378e51ae5adf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:50.804130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf88990-b45a-49e3-8ee6-378e51ae5adf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:50.810611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330876d3-f953-4808-ba30-73605ce05c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:50.811500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330876d3-f953-4808-ba30-73605ce05c02 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.811950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330876d3-f953-4808-ba30-73605ce05c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.812370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330876d3-f953-4808-ba30-73605ce05c02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.842941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 57/109] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:28:50 2022] POST /identity/v3/projects => generated 302 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:28:50.845079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652de01-d13c-4fe6-8a89-0f30f93a639e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.845517 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652de01-d13c-4fe6-8a89-0f30f93a639e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.845998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652de01-d13c-4fe6-8a89-0f30f93a639e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.869624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-1652de01-d13c-4fe6-8a89-0f30f93a639e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:50.871247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1652de01-d13c-4fe6-8a89-0f30f93a639e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:50.880348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:50.949574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330876d3-f953-4808-ba30-73605ce05c02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:50.951321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330876d3-f953-4808-ba30-73605ce05c02 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:50.957251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 58/110] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:50.959418 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0d99f-1e62-48e6-9fae-517dd411f96a None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.959926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0d99f-1e62-48e6-9fae-517dd411f96a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.960369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f0d99f-1e62-48e6-9fae-517dd411f96a None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.962123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 59/111] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:50.970398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961683-d338-46e6-9b0b-c9d5f5f06236 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:50.970993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961683-d338-46e6-9b0b-c9d5f5f06236 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:50.971554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a961683-d338-46e6-9b0b-c9d5f5f06236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:50.997707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-8a961683-d338-46e6-9b0b-c9d5f5f06236 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:50.999285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8a961683-d338-46e6-9b0b-c9d5f5f06236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:51.082292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 60/112] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:51.083179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 53/113] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:28:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:28:51.088725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a961683-d338-46e6-9b0b-c9d5f5f06236 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:51.089977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330876d3-f953-4808-ba30-73605ce05c02 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:51.100733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:51.169738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:51.203373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a961683-d338-46e6-9b0b-c9d5f5f06236 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:51.240175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42627e-7031-46bc-b358-42cb3349bd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:51.241162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42627e-7031-46bc-b358-42cb3349bd36 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:51.241577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42627e-7031-46bc-b358-42cb3349bd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:51.241986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e42627e-7031-46bc-b358-42cb3349bd36 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:51.253111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e42627e-7031-46bc-b358-42cb3349bd36 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-1e42627e-7031-46bc-b358-42cb3349bd36 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:51.259202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:51.260238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:51.261243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:28:51.263319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 61/114] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:51.269904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42627e-7031-46bc-b358-42cb3349bd36 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:51.287012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330876d3-f953-4808-ba30-73605ce05c02 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:51.343118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92f6a6-ddf0-49f0-a220-5183d0ca3795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:51.343801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92f6a6-ddf0-49f0-a220-5183d0ca3795 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:51.344033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92f6a6-ddf0-49f0-a220-5183d0ca3795 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:51.344626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed92f6a6-ddf0-49f0-a220-5183d0ca3795 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:51.345587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92f6a6-ddf0-49f0-a220-5183d0ca3795 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:51.347068 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed92f6a6-ddf0-49f0-a220-5183d0ca3795 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:51.349250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 54/115] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:51 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:51.350801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92f6a6-ddf0-49f0-a220-5183d0ca3795 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:51.403531 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e42627e-7031-46bc-b358-42cb3349bd36 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:51.441296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:51.441506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:51.449976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b7082-ca1b-46dd-9ca4-68ecab8c6c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:51.450956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b7082-ca1b-46dd-9ca4-68ecab8c6c27 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:51.451372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b7082-ca1b-46dd-9ca4-68ecab8c6c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:51.451780 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687b7082-ca1b-46dd-9ca4-68ecab8c6c27 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:51.452772 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b7082-ca1b-46dd-9ca4-68ecab8c6c27 None admin] RBAC: Adding query filter params (name=member) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:51.453213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b7082-ca1b-46dd-9ca4-68ecab8c6c27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:51.455011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687b7082-ca1b-46dd-9ca4-68ecab8c6c27 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:51.467069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 62/116] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:28:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:51.468208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b7082-ca1b-46dd-9ca4-68ecab8c6c27 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:51.472450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed92f6a6-ddf0-49f0-a220-5183d0ca3795 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:51.510050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c736a59-9689-4002-bb7d-3ed8e7665cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:51.510606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c736a59-9689-4002-bb7d-3ed8e7665cf3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:51.510891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c736a59-9689-4002-bb7d-3ed8e7665cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:51.511169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c736a59-9689-4002-bb7d-3ed8e7665cf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:51.521719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c736a59-9689-4002-bb7d-3ed8e7665cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-5c736a59-9689-4002-bb7d-3ed8e7665cf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:51.527033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:51.527934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:28:51.528681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 55/117] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:28:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:51.529965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c736a59-9689-4002-bb7d-3ed8e7665cf3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:51.588194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687b7082-ca1b-46dd-9ca4-68ecab8c6c27 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:51.623405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e1b06-ec21-48b8-b9f7-955fccb37118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:51.624010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e1b06-ec21-48b8-b9f7-955fccb37118 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:51.624294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e1b06-ec21-48b8-b9f7-955fccb37118 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:51.624563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e1b06-ec21-48b8-b9f7-955fccb37118 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:51.625319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e1b06-ec21-48b8-b9f7-955fccb37118 None admin] RBAC: Adding query filter params (role.id=4f87bf3cce2d4008a95c1da7369d74bf, scope.project.id=a6cc40a4154043ada48161d66116e3c8, user.id=2ab0da9f2e6f43dc9337b9fc7bc13c2a) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:51.625624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e1b06-ec21-48b8-b9f7-955fccb37118 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:51.627041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e1b06-ec21-48b8-b9f7-955fccb37118 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:51.635175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 63/118] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:28:51 2022] GET /identity/v3/role_assignments?user.id=2ab0da9f2e6f43dc9337b9fc7bc13c2a&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=4f87bf3cce2d4008a95c1da7369d74bf => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:51.636496 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d3be9-a174-4928-84fd-ef780b08a1b5 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:51.636803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d3be9-a174-4928-84fd-ef780b08a1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:51.637087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056d3be9-a174-4928-84fd-ef780b08a1b5 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:51.638288 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 64/119] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:51.639076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d3be9-a174-4928-84fd-ef780b08a1b5 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:51.645172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c736a59-9689-4002-bb7d-3ed8e7665cf3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:51.700165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26828e6b-b622-46be-8759-4e132b28cf88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:51.701326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26828e6b-b622-46be-8759-4e132b28cf88 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:51.702501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26828e6b-b622-46be-8759-4e132b28cf88 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:51.703069 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26828e6b-b622-46be-8759-4e132b28cf88 None admin] PATH_INFO: `/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:51.705127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26828e6b-b622-46be-8759-4e132b28cf88 None admin] RBAC: Authorizing `identity:get_user(user_id=0cac9a0d6d824acfbc13df63c8e5dff8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:51.707642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26828e6b-b622-46be-8759-4e132b28cf88 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:51.708309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:51.712794 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 56/120] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:51 2022] GET /identity/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:51.714768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26828e6b-b622-46be-8759-4e132b28cf88 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:51.720716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:51.766512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056d3be9-a174-4928-84fd-ef780b08a1b5 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:51.811319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6af9acb-acd5-4f70-bcec-018666a9df5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:51.812372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af9acb-acd5-4f70-bcec-018666a9df5b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:51.812929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af9acb-acd5-4f70-bcec-018666a9df5b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:51.813255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6af9acb-acd5-4f70-bcec-018666a9df5b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:51.825241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6af9acb-acd5-4f70-bcec-018666a9df5b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-b6af9acb-acd5-4f70-bcec-018666a9df5b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:51.831740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:51.832614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:28:51.833409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 65/121] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:28:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:51.835178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a597d-1928-4bfc-8ac3-c84cfb4c749e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:51.835370 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a597d-1928-4bfc-8ac3-c84cfb4c749e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:51.836877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533a597d-1928-4bfc-8ac3-c84cfb4c749e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:51.843175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26828e6b-b622-46be-8759-4e132b28cf88 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:51.854224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-533a597d-1928-4bfc-8ac3-c84cfb4c749e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:51.855665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-533a597d-1928-4bfc-8ac3-c84cfb4c749e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:51.884898 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fa37c-3327-4a90-8046-3e635a5eefbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:51.886214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fa37c-3327-4a90-8046-3e635a5eefbb None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:51.886941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fa37c-3327-4a90-8046-3e635a5eefbb None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:51.887396 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206fa37c-3327-4a90-8046-3e635a5eefbb None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:51.888494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206fa37c-3327-4a90-8046-3e635a5eefbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:51.889301 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206fa37c-3327-4a90-8046-3e635a5eefbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:51.891320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206fa37c-3327-4a90-8046-3e635a5eefbb None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:51.902165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 57/122] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:51.904791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fa37c-3327-4a90-8046-3e635a5eefbb None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:51.933987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 66/123] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:51.940800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a597d-1928-4bfc-8ac3-c84cfb4c749e None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:51.993134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:52.013817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206fa37c-3327-4a90-8046-3e635a5eefbb None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:52.027178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:52.050938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f239c7a-cd8e-48eb-b234-9510301c3169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:52.051762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239c7a-cd8e-48eb-b234-9510301c3169 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:52.052150 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239c7a-cd8e-48eb-b234-9510301c3169 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:52.052775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f239c7a-cd8e-48eb-b234-9510301c3169 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:52.054110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f239c7a-cd8e-48eb-b234-9510301c3169 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:52.055695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f239c7a-cd8e-48eb-b234-9510301c3169 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:52.057493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 58/124] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:51 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:52.058365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f239c7a-cd8e-48eb-b234-9510301c3169 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:52.116800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533a597d-1928-4bfc-8ac3-c84cfb4c749e None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:52.160587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f239c7a-cd8e-48eb-b234-9510301c3169 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:52.198288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eba1af-98c9-458f-8c82-708034a19dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:52.199201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eba1af-98c9-458f-8c82-708034a19dbd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:52.199665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eba1af-98c9-458f-8c82-708034a19dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:52.200050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eba1af-98c9-458f-8c82-708034a19dbd None admin] PATH_INFO: `/v3/users/fee2283327204c6f8ee811e2c566e379` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:52.201595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eba1af-98c9-458f-8c82-708034a19dbd None admin] RBAC: Authorizing `identity:get_user(user_id=fee2283327204c6f8ee811e2c566e379)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:52.203236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eba1af-98c9-458f-8c82-708034a19dbd None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:52.205467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 59/125] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:52 2022] GET /identity/v3/users/fee2283327204c6f8ee811e2c566e379 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:52.206308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eba1af-98c9-458f-8c82-708034a19dbd None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:52.206469 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44a336-2a12-49f4-8992-fea9feb6859d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:52.207241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44a336-2a12-49f4-8992-fea9feb6859d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:52.207648 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44a336-2a12-49f4-8992-fea9feb6859d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:52.208045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c44a336-2a12-49f4-8992-fea9feb6859d None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:52.209381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44a336-2a12-49f4-8992-fea9feb6859d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:52.209812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44a336-2a12-49f4-8992-fea9feb6859d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:52.211403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c44a336-2a12-49f4-8992-fea9feb6859d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:52.222767 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:52.234626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 67/126] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:28:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:52.235686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44a336-2a12-49f4-8992-fea9feb6859d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:52.323765 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eba1af-98c9-458f-8c82-708034a19dbd None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:52.359458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48a052-50a1-4384-aa9d-26d2655a5c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:52.360287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48a052-50a1-4384-aa9d-26d2655a5c8d None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:52.360669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48a052-50a1-4384-aa9d-26d2655a5c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:52.361076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48a052-50a1-4384-aa9d-26d2655a5c8d None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8/users/0cac9a0d6d824acfbc13df63c8e5dff8/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:52.363179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48a052-50a1-4384-aa9d-26d2655a5c8d None admin] RBAC: Authorizing `identity:create_grant(project_id=a6cc40a4154043ada48161d66116e3c8, user_id=0cac9a0d6d824acfbc13df63c8e5dff8, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:52.365455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48a052-50a1-4384-aa9d-26d2655a5c8d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:52.388787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 60/127] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:28:52 2022] PUT /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8/users/0cac9a0d6d824acfbc13df63c8e5dff8/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:28:52.388787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48a052-50a1-4384-aa9d-26d2655a5c8d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:52.432749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c44a336-2a12-49f4-8992-fea9feb6859d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:52.449371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:52.500929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1ae83-f9dd-40a5-b658-f7dcd6678d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:52.501668 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1ae83-f9dd-40a5-b658-f7dcd6678d03 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:52.502686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1ae83-f9dd-40a5-b658-f7dcd6678d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:52.502993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a1ae83-f9dd-40a5-b658-f7dcd6678d03 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:52.506555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1ae83-f9dd-40a5-b658-f7dcd6678d03 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:52.509262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a1ae83-f9dd-40a5-b658-f7dcd6678d03 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:52.511814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 68/128] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:52 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:52.513151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1ae83-f9dd-40a5-b658-f7dcd6678d03 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:52.515027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:52.603127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48a052-50a1-4384-aa9d-26d2655a5c8d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:52.652574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca2f49-1c5d-422d-8bf9-74a3654882d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:52.653879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2f49-1c5d-422d-8bf9-74a3654882d8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:52.654382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2f49-1c5d-422d-8bf9-74a3654882d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:52.655154 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ca2f49-1c5d-422d-8bf9-74a3654882d8 None admin] PATH_INFO: `/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:52.661120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca2f49-1c5d-422d-8bf9-74a3654882d8 None admin] RBAC: Authorizing `identity:get_user(user_id=0cac9a0d6d824acfbc13df63c8e5dff8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:52.663969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ca2f49-1c5d-422d-8bf9-74a3654882d8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:52.666856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 61/129] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:52 2022] GET /identity/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8 => generated 275 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:52.672488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca2f49-1c5d-422d-8bf9-74a3654882d8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:52.703684 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a1ae83-f9dd-40a5-b658-f7dcd6678d03 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:52.751043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7c238-47cc-4362-938d-f7c4bb14feef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:52.752302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7c238-47cc-4362-938d-f7c4bb14feef None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:52.752837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7c238-47cc-4362-938d-f7c4bb14feef None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:52.753422 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff7c238-47cc-4362-938d-f7c4bb14feef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:52.767624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff7c238-47cc-4362-938d-f7c4bb14feef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-5ff7c238-47cc-4362-938d-f7c4bb14feef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:52.775508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:52.776438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:52.777247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:28:52.778250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 69/130] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:52.778792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7c238-47cc-4362-938d-f7c4bb14feef None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:52.791345 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ca2f49-1c5d-422d-8bf9-74a3654882d8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:52.821495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:52.830316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:52.834570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a657-5204-40ef-ab6d-bd8f79763621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:52.836455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a657-5204-40ef-ab6d-bd8f79763621 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:52.837026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a657-5204-40ef-ab6d-bd8f79763621 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:52.837503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f4a657-5204-40ef-ab6d-bd8f79763621 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8/users/fee2283327204c6f8ee811e2c566e379/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:52.840995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4a657-5204-40ef-ab6d-bd8f79763621 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6cc40a4154043ada48161d66116e3c8, user_id=fee2283327204c6f8ee811e2c566e379, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:52.843401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f4a657-5204-40ef-ab6d-bd8f79763621 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:52.875532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 62/131] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:28:52 2022] PUT /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8/users/fee2283327204c6f8ee811e2c566e379/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:28:52.876691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a657-5204-40ef-ab6d-bd8f79763621 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:52.918976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff7c238-47cc-4362-938d-f7c4bb14feef None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:52.964429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7129c9-7a5e-4c0c-b231-859785165da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:52.965129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7129c9-7a5e-4c0c-b231-859785165da5 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:52.965414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7129c9-7a5e-4c0c-b231-859785165da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:52.965758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7129c9-7a5e-4c0c-b231-859785165da5 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:52.967122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7129c9-7a5e-4c0c-b231-859785165da5 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:52.968959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7129c9-7a5e-4c0c-b231-859785165da5 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:52.971125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 70/132] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:52 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:52.995061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7129c9-7a5e-4c0c-b231-859785165da5 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:53.050634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f4a657-5204-40ef-ab6d-bd8f79763621 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:53.088979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859a4bb-a9dc-44eb-b263-dd4a5dc5d3d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:53.089817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859a4bb-a9dc-44eb-b263-dd4a5dc5d3d8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:53.090244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859a4bb-a9dc-44eb-b263-dd4a5dc5d3d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:53.090683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859a4bb-a9dc-44eb-b263-dd4a5dc5d3d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:53.091507 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859a4bb-a9dc-44eb-b263-dd4a5dc5d3d8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:53.092017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859a4bb-a9dc-44eb-b263-dd4a5dc5d3d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:53.093573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6859a4bb-a9dc-44eb-b263-dd4a5dc5d3d8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:53.103240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 63/133] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:28:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:53.108845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859a4bb-a9dc-44eb-b263-dd4a5dc5d3d8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:53.170195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:53.222687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6859a4bb-a9dc-44eb-b263-dd4a5dc5d3d8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:53.261044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8fcac-4821-44d9-8885-03fab289fe0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:53.261932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8fcac-4821-44d9-8885-03fab289fe0c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:53.262383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8fcac-4821-44d9-8885-03fab289fe0c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:53.262798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8fcac-4821-44d9-8885-03fab289fe0c None admin] PATH_INFO: `/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:53.264124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8fcac-4821-44d9-8885-03fab289fe0c None admin] RBAC: Authorizing `identity:get_user(user_id=0cac9a0d6d824acfbc13df63c8e5dff8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:53.265766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb8fcac-4821-44d9-8885-03fab289fe0c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:53.267521 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 64/134] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:53 2022] GET /identity/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:53.269835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7129c9-7a5e-4c0c-b231-859785165da5 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:53.273269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8fcac-4821-44d9-8885-03fab289fe0c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:53.291076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:53.353902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93934dba-226d-4096-ad72-0c9acb19fcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:53.354914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93934dba-226d-4096-ad72-0c9acb19fcc9 None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:53.355396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93934dba-226d-4096-ad72-0c9acb19fcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:53.355822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93934dba-226d-4096-ad72-0c9acb19fcc9 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8/users/0cac9a0d6d824acfbc13df63c8e5dff8/roles/952084593c184a78a952e1797a0b6c6b` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:53.361177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93934dba-226d-4096-ad72-0c9acb19fcc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6cc40a4154043ada48161d66116e3c8, user_id=0cac9a0d6d824acfbc13df63c8e5dff8, role_id=952084593c184a78a952e1797a0b6c6b)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:53.362839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93934dba-226d-4096-ad72-0c9acb19fcc9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:53.395885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 71/135] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:28:52 2022] PUT /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8/users/0cac9a0d6d824acfbc13df63c8e5dff8/roles/952084593c184a78a952e1797a0b6c6b => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:28:53.436277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8fcac-4821-44d9-8885-03fab289fe0c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:53.459467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:53.476131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59acbf7-0623-41ec-a1c0-2230b06ad646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:53.476868 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59acbf7-0623-41ec-a1c0-2230b06ad646 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:53.477154 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59acbf7-0623-41ec-a1c0-2230b06ad646 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:53.477429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59acbf7-0623-41ec-a1c0-2230b06ad646 None admin] PATH_INFO: `/v3/projects/929f910a2e0d4acc995c08172ce891f3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:53.478595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59acbf7-0623-41ec-a1c0-2230b06ad646 None admin] RBAC: Authorizing `identity:get_project(project_id=929f910a2e0d4acc995c08172ce891f3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:53.480072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59acbf7-0623-41ec-a1c0-2230b06ad646 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:53.481777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 65/136] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:53 2022] GET /identity/v3/projects/929f910a2e0d4acc995c08172ce891f3 => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:53.487836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93934dba-226d-4096-ad72-0c9acb19fcc9 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:53.629287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93934dba-226d-4096-ad72-0c9acb19fcc9 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:53.679471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3564ebbe-3748-4303-b257-3cd8feb0a074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:53.680154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ebbe-3748-4303-b257-3cd8feb0a074 None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:53.680435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ebbe-3748-4303-b257-3cd8feb0a074 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:53.680712 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3564ebbe-3748-4303-b257-3cd8feb0a074 None admin] PATH_INFO: `/v3/projects/929f910a2e0d4acc995c08172ce891f3/users/0cac9a0d6d824acfbc13df63c8e5dff8/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:53.682767 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564ebbe-3748-4303-b257-3cd8feb0a074 None admin] RBAC: Authorizing `identity:create_grant(project_id=929f910a2e0d4acc995c08172ce891f3, user_id=0cac9a0d6d824acfbc13df63c8e5dff8, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:53.684290 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3564ebbe-3748-4303-b257-3cd8feb0a074 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:53.712969 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 72/137] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:28:53 2022] PUT /identity/v3/projects/929f910a2e0d4acc995c08172ce891f3/users/0cac9a0d6d824acfbc13df63c8e5dff8/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:28:54.052731 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4b1cf-39e9-4774-8b18-44f2ea0f8c95 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:54.053113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4b1cf-39e9-4774-8b18-44f2ea0f8c95 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:54.053274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4b1cf-39e9-4774-8b18-44f2ea0f8c95 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:54.054850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 66/138] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:54.060375 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1f7f6-770d-4deb-94e8-edca6e0a7801 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:54.060669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1f7f6-770d-4deb-94e8-edca6e0a7801 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:54.060973 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df1f7f6-770d-4deb-94e8-edca6e0a7801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:54.080958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2df1f7f6-770d-4deb-94e8-edca6e0a7801 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:54.082171 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2df1f7f6-770d-4deb-94e8-edca6e0a7801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:54.110114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:54.174130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 73/139] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:54.465855 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5927c-3e72-46e0-a87f-39e3f8019706 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:54.466878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5927c-3e72-46e0-a87f-39e3f8019706 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:54.467363 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde5927c-3e72-46e0-a87f-39e3f8019706 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:54.468938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 67/140] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:54.474568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff3e1-2387-429a-af8d-38933f31f4a9 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:54.475252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff3e1-2387-429a-af8d-38933f31f4a9 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:54.475755 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feff3e1-2387-429a-af8d-38933f31f4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:54.498873 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-3feff3e1-2387-429a-af8d-38933f31f4a9 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:54.500817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3feff3e1-2387-429a-af8d-38933f31f4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:54.595416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 74/141] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:54.700253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5927c-3e72-46e0-a87f-39e3f8019706 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:54.830489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde5927c-3e72-46e0-a87f-39e3f8019706 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:54.874500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67cc1c-7998-44d8-a491-395e87ba1b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:54.875133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67cc1c-7998-44d8-a491-395e87ba1b47 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:54.875409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67cc1c-7998-44d8-a491-395e87ba1b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:54.875680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe67cc1c-7998-44d8-a491-395e87ba1b47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:54.876805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67cc1c-7998-44d8-a491-395e87ba1b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:54.878236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe67cc1c-7998-44d8-a491-395e87ba1b47 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:54.879793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 68/142] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:28:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:54.886173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feff3e1-2387-429a-af8d-38933f31f4a9 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:54.982762 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:55.000533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feff3e1-2387-429a-af8d-38933f31f4a9 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:55.038594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a71f76-4921-4809-bd20-0e413efc508c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:55.039210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a71f76-4921-4809-bd20-0e413efc508c None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:55.039487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a71f76-4921-4809-bd20-0e413efc508c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:55.039754 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a71f76-4921-4809-bd20-0e413efc508c None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:55.040423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a71f76-4921-4809-bd20-0e413efc508c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:55.041917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a71f76-4921-4809-bd20-0e413efc508c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:55.065632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 75/143] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:28:54 2022] POST /identity/v3/users => generated 283 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:28:55.141181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67cc1c-7998-44d8-a491-395e87ba1b47 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:55.208951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:55.274070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe67cc1c-7998-44d8-a491-395e87ba1b47 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:55.321030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff4525-cae4-4a76-bbf1-223e89e12e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:55.321670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff4525-cae4-4a76-bbf1-223e89e12e81 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:55.321944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff4525-cae4-4a76-bbf1-223e89e12e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:55.322210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ff4525-cae4-4a76-bbf1-223e89e12e81 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:55.335410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ff4525-cae4-4a76-bbf1-223e89e12e81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-93ff4525-cae4-4a76-bbf1-223e89e12e81 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:55.340812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:55.341676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:28:55.342457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 69/144] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:55.347918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a71f76-4921-4809-bd20-0e413efc508c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:55.468678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a71f76-4921-4809-bd20-0e413efc508c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:55.509445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca78866-c92f-44b2-8f99-fd0c346f4e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:55.510073 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca78866-c92f-44b2-8f99-fd0c346f4e37 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:55.510319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca78866-c92f-44b2-8f99-fd0c346f4e37 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:55.510584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca78866-c92f-44b2-8f99-fd0c346f4e37 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:55.511244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca78866-c92f-44b2-8f99-fd0c346f4e37 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:55.511539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca78866-c92f-44b2-8f99-fd0c346f4e37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:55.512936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca78866-c92f-44b2-8f99-fd0c346f4e37 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:55.522185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 76/145] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:28:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:55.527981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff4525-cae4-4a76-bbf1-223e89e12e81 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:55.695636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ff4525-cae4-4a76-bbf1-223e89e12e81 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:55.714553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:55.738500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd62b86-b22c-491d-afb6-60681a7fdb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:55.739047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd62b86-b22c-491d-afb6-60681a7fdb26 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:55.739321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd62b86-b22c-491d-afb6-60681a7fdb26 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:55.739587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd62b86-b22c-491d-afb6-60681a7fdb26 None admin] PATH_INFO: `/v3/users/2ab0da9f2e6f43dc9337b9fc7bc13c2a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:55.740933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd62b86-b22c-491d-afb6-60681a7fdb26 None admin] RBAC: Authorizing `identity:get_user(user_id=2ab0da9f2e6f43dc9337b9fc7bc13c2a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:55.742618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd62b86-b22c-491d-afb6-60681a7fdb26 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:55.744365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 70/146] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:55 2022] GET /identity/v3/users/2ab0da9f2e6f43dc9337b9fc7bc13c2a => generated 289 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:55.750653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca78866-c92f-44b2-8f99-fd0c346f4e37 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:55.754504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:55.761840 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f42984-7a22-49bc-9feb-8fa530bd6100 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:55.762177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f42984-7a22-49bc-9feb-8fa530bd6100 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:55.762469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f42984-7a22-49bc-9feb-8fa530bd6100 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:55.763783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 71/147] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:28:55.769066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460319bb-d54d-47c3-8441-d0144285d1b5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:55.769401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460319bb-d54d-47c3-8441-d0144285d1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:55.769692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460319bb-d54d-47c3-8441-d0144285d1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:55.788934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-460319bb-d54d-47c3-8441-d0144285d1b5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:55.790464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-460319bb-d54d-47c3-8441-d0144285d1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:55.861920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 72/148] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:28:55.906976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca78866-c92f-44b2-8f99-fd0c346f4e37 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:55.956224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd5744f-f863-465b-9be8-9a43f8efdb10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:55.956894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5744f-f863-465b-9be8-9a43f8efdb10 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:55.957174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5744f-f863-465b-9be8-9a43f8efdb10 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:55.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd5744f-f863-465b-9be8-9a43f8efdb10 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:55.959145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5744f-f863-465b-9be8-9a43f8efdb10 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:55.960573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd5744f-f863-465b-9be8-9a43f8efdb10 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:55.962525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 77/149] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:55 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:55.968227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460319bb-d54d-47c3-8441-d0144285d1b5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:56.041238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771129b-02e5-4933-9d8f-e51d93e3ef44 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:56.041523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771129b-02e5-4933-9d8f-e51d93e3ef44 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:56.041698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0771129b-02e5-4933-9d8f-e51d93e3ef44 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:56.043045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 78/150] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:28:56.048254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48346a-83b8-417a-86a5-79bf83e76a58 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:56.048533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48346a-83b8-417a-86a5-79bf83e76a58 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:56.048823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c48346a-83b8-417a-86a5-79bf83e76a58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:56.085849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-0c48346a-83b8-417a-86a5-79bf83e76a58 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:56.087044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-0c48346a-83b8-417a-86a5-79bf83e76a58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:56.108295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460319bb-d54d-47c3-8441-d0144285d1b5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:56.129778 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:56.174937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfc769-b542-4c1b-ade3-2847acb12fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:56.175609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfc769-b542-4c1b-ade3-2847acb12fa2 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:56.175880 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfc769-b542-4c1b-ade3-2847acb12fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:56.176147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cfc769-b542-4c1b-ade3-2847acb12fa2 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8/users/2ab0da9f2e6f43dc9337b9fc7bc13c2a/roles/4f87bf3cce2d4008a95c1da7369d74bf` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:56.178824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfc769-b542-4c1b-ade3-2847acb12fa2 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6cc40a4154043ada48161d66116e3c8, user_id=2ab0da9f2e6f43dc9337b9fc7bc13c2a, role_id=4f87bf3cce2d4008a95c1da7369d74bf)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:56.180288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cfc769-b542-4c1b-ade3-2847acb12fa2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:56.204206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 73/151] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:28:55 2022] PUT /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8/users/2ab0da9f2e6f43dc9337b9fc7bc13c2a/roles/4f87bf3cce2d4008a95c1da7369d74bf => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:28:56.243681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 79/152] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:28:56.289273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfc769-b542-4c1b-ade3-2847acb12fa2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:56.423151 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cfc769-b542-4c1b-ade3-2847acb12fa2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:56.464972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc10c5a-85ad-432b-a533-73f4822ef38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:56.465619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc10c5a-85ad-432b-a533-73f4822ef38c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:56.465891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc10c5a-85ad-432b-a533-73f4822ef38c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:56.466159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc10c5a-85ad-432b-a533-73f4822ef38c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:56.479035 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc10c5a-85ad-432b-a533-73f4822ef38c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-0fc10c5a-85ad-432b-a533-73f4822ef38c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:56.484488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:56.485347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:28:56.486089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 74/153] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:56.493906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48346a-83b8-417a-86a5-79bf83e76a58 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:56.496901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:56.657514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c48346a-83b8-417a-86a5-79bf83e76a58 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:56.698005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd32eae-1757-42ce-a7a0-fae5a962f225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:56.698547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd32eae-1757-42ce-a7a0-fae5a962f225 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:56.698827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd32eae-1757-42ce-a7a0-fae5a962f225 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:56.699101 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd32eae-1757-42ce-a7a0-fae5a962f225 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:56.699774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd32eae-1757-42ce-a7a0-fae5a962f225 None admin] RBAC: Adding query filter params (name=member) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:56.700076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd32eae-1757-42ce-a7a0-fae5a962f225 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:56.701480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd32eae-1757-42ce-a7a0-fae5a962f225 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:56.711352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 80/154] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:28:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:56.717359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc10c5a-85ad-432b-a533-73f4822ef38c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:56.769338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:56.816191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd32eae-1757-42ce-a7a0-fae5a962f225 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:56.856166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc10c5a-85ad-432b-a533-73f4822ef38c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:56.900553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511b654-82c3-4292-82b2-093449ce54af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:56.901548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511b654-82c3-4292-82b2-093449ce54af None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:56.904881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511b654-82c3-4292-82b2-093449ce54af None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:56.904881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511b654-82c3-4292-82b2-093449ce54af None admin] PATH_INFO: `/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:56.904881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511b654-82c3-4292-82b2-093449ce54af None admin] RBAC: Authorizing `identity:get_user(user_id=0cac9a0d6d824acfbc13df63c8e5dff8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:56.905246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f511b654-82c3-4292-82b2-093449ce54af None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:56.908045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 75/155] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:56 2022] GET /identity/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:56.918699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511b654-82c3-4292-82b2-093449ce54af None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:57.030242 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:57.069700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511b654-82c3-4292-82b2-093449ce54af None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:57.071995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:57.123776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbfa8d7-bb8d-462a-958c-aa140f08c481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:57.124392 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfa8d7-bb8d-462a-958c-aa140f08c481 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:57.124666 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfa8d7-bb8d-462a-958c-aa140f08c481 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:57.125213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbfa8d7-bb8d-462a-958c-aa140f08c481 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:57.174814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd32eae-1757-42ce-a7a0-fae5a962f225 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:57.209444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfa8d7-bb8d-462a-958c-aa140f08c481 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:57.211176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbfa8d7-bb8d-462a-958c-aa140f08c481 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:57.245497 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7154e70-a887-41ed-8b12-2c5880d93586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:57.246317 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7154e70-a887-41ed-8b12-2c5880d93586 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:57.246695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7154e70-a887-41ed-8b12-2c5880d93586 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:57.247195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7154e70-a887-41ed-8b12-2c5880d93586 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:57.266664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7154e70-a887-41ed-8b12-2c5880d93586 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-b7154e70-a887-41ed-8b12-2c5880d93586 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:57.272735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:57.273889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:28:57.274694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:28:57.281625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 81/156] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:28:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 466 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:57.284386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eee5e2-3fbf-440a-9f0d-8c9132dcc570 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:57.285246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eee5e2-3fbf-440a-9f0d-8c9132dcc570 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:57.286023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94eee5e2-3fbf-440a-9f0d-8c9132dcc570 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:57.293339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 82/157] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:57 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:57.296577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eee5e2-3fbf-440a-9f0d-8c9132dcc570 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:57.320787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 76/158] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:28:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:28:57.320969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c79c82-7f1a-4e43-8f97-f4abc5306eb7 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:57.321216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c79c82-7f1a-4e43-8f97-f4abc5306eb7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:57.321413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c79c82-7f1a-4e43-8f97-f4abc5306eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:57.341963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-76c79c82-7f1a-4e43-8f97-f4abc5306eb7 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:57.343172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-76c79c82-7f1a-4e43-8f97-f4abc5306eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:57.378772 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:57.419160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 77/159] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:57.420269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c79c82-7f1a-4e43-8f97-f4abc5306eb7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:57.536800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c79c82-7f1a-4e43-8f97-f4abc5306eb7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:57.575868 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:57.585204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b6eb3-9e24-439c-ad6f-401c08cab11e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:57.585997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b6eb3-9e24-439c-ad6f-401c08cab11e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:57.586277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b6eb3-9e24-439c-ad6f-401c08cab11e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:57.586541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b6eb3-9e24-439c-ad6f-401c08cab11e None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:57.587792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b6eb3-9e24-439c-ad6f-401c08cab11e None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:57.589309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b6eb3-9e24-439c-ad6f-401c08cab11e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:57.589417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94eee5e2-3fbf-440a-9f0d-8c9132dcc570 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:57.591007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 78/160] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:57 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:57.596928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b6eb3-9e24-439c-ad6f-401c08cab11e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:57.623535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:57.661586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d29d59-6323-4b07-9147-a4562a1a28b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:57.662570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d29d59-6323-4b07-9147-a4562a1a28b8 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:57.662762 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d29d59-6323-4b07-9147-a4562a1a28b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:57.662998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d29d59-6323-4b07-9147-a4562a1a28b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:57.663699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d29d59-6323-4b07-9147-a4562a1a28b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:57.663962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d29d59-6323-4b07-9147-a4562a1a28b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:57.665369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d29d59-6323-4b07-9147-a4562a1a28b8 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:57.674605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 83/161] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:57.684327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d29d59-6323-4b07-9147-a4562a1a28b8 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:57.834729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d29d59-6323-4b07-9147-a4562a1a28b8 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:57.892295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674e34d7-875c-494f-87d8-bc12dfdc57f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:57.893327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e34d7-875c-494f-87d8-bc12dfdc57f5 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:57.893817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e34d7-875c-494f-87d8-bc12dfdc57f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:57.894374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e34d7-875c-494f-87d8-bc12dfdc57f5 None admin] PATH_INFO: `/v3/users/fee2283327204c6f8ee811e2c566e379` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:57.895738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e34d7-875c-494f-87d8-bc12dfdc57f5 None admin] RBAC: Authorizing `identity:get_user(user_id=fee2283327204c6f8ee811e2c566e379)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:57.897663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-674e34d7-875c-494f-87d8-bc12dfdc57f5 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:57.905105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 84/162] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:57 2022] GET /identity/v3/users/fee2283327204c6f8ee811e2c566e379 => generated 247 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:57.907068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df882056-dde6-4c78-b91c-d9903efdc7b5 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:57.907553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df882056-dde6-4c78-b91c-d9903efdc7b5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:57.908029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df882056-dde6-4c78-b91c-d9903efdc7b5 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:57.912110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 85/163] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:57.913611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df882056-dde6-4c78-b91c-d9903efdc7b5 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:57.924962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b6eb3-9e24-439c-ad6f-401c08cab11e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:58.015331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a371dc6a-0355-47fe-9834-7da87425c5bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:58.015331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371dc6a-0355-47fe-9834-7da87425c5bc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:58.015331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371dc6a-0355-47fe-9834-7da87425c5bc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:58.015674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a371dc6a-0355-47fe-9834-7da87425c5bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:58.016247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371dc6a-0355-47fe-9834-7da87425c5bc None admin] RBAC: Adding query filter params (role.id=aac9ca64da09463ba9b3966c7188d7ee, scope.project.id=a6cc40a4154043ada48161d66116e3c8, user.id=0cac9a0d6d824acfbc13df63c8e5dff8) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:58.016553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371dc6a-0355-47fe-9834-7da87425c5bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:58.018064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a371dc6a-0355-47fe-9834-7da87425c5bc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:58.019960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:58.034900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 79/164] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:28:57 2022] GET /identity/v3/role_assignments?user.id=0cac9a0d6d824acfbc13df63c8e5dff8&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 608 bytes in 438 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:58.036138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568ac53-3bbb-40df-900e-384af683d890 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:58.036421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568ac53-3bbb-40df-900e-384af683d890 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:58.036421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6568ac53-3bbb-40df-900e-384af683d890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:58.077057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6568ac53-3bbb-40df-900e-384af683d890 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:28:58.078268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6568ac53-3bbb-40df-900e-384af683d890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:28:58.098220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df882056-dde6-4c78-b91c-d9903efdc7b5 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:58.158502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e8d40-a84a-4392-935c-a9d29395fac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:58.159135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e8d40-a84a-4392-935c-a9d29395fac4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:58.159406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e8d40-a84a-4392-935c-a9d29395fac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:58.159665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e8d40-a84a-4392-935c-a9d29395fac4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:58.242455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:58.267486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 80/165] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:28:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:28:58.268425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568ac53-3bbb-40df-900e-384af683d890 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:58.309929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e8d40-a84a-4392-935c-a9d29395fac4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:58.311434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e8d40-a84a-4392-935c-a9d29395fac4 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:58.421304 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:58.461256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6568ac53-3bbb-40df-900e-384af683d890 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:58.506881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2cb8f-8532-447c-8e15-c008f868fbef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:58.508166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2cb8f-8532-447c-8e15-c008f868fbef None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:58.508643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2cb8f-8532-447c-8e15-c008f868fbef None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:58.509159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2cb8f-8532-447c-8e15-c008f868fbef None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:58.532020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 86/166] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:28:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:28:58.533076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2cb8f-8532-447c-8e15-c008f868fbef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:58.538650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e8d40-a84a-4392-935c-a9d29395fac4 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-8dd2cb8f-8532-447c-8e15-c008f868fbef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:58.543971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:58.545025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:28:58.545936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:28:58.555262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 81/167] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:28:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:58.567592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2cb8f-8532-447c-8e15-c008f868fbef None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:58.696251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e8d40-a84a-4392-935c-a9d29395fac4 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:58.719634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:58.758708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba2dfe-5d5a-4f88-b708-28176ee9a5f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:58.759738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba2dfe-5d5a-4f88-b708-28176ee9a5f6 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:58.760239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba2dfe-5d5a-4f88-b708-28176ee9a5f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:58.760708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eba2dfe-5d5a-4f88-b708-28176ee9a5f6 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:58.762079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba2dfe-5d5a-4f88-b708-28176ee9a5f6 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:58.764240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eba2dfe-5d5a-4f88-b708-28176ee9a5f6 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:58.766172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 87/168] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:58 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:58.772713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba2dfe-5d5a-4f88-b708-28176ee9a5f6 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:58.804062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:58.852749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2cb8f-8532-447c-8e15-c008f868fbef None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:58.899409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eba2dfe-5d5a-4f88-b708-28176ee9a5f6 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:58.926473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab91fc3-eee2-49d7-9db4-1b61fb683351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:58.928405 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab91fc3-eee2-49d7-9db4-1b61fb683351 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:58.928943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab91fc3-eee2-49d7-9db4-1b61fb683351 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:58.929428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab91fc3-eee2-49d7-9db4-1b61fb683351 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:58.930420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab91fc3-eee2-49d7-9db4-1b61fb683351 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:58.930936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab91fc3-eee2-49d7-9db4-1b61fb683351 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:58.932539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab91fc3-eee2-49d7-9db4-1b61fb683351 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:58.947548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 82/169] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:28:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 376 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:28:58.949270 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab91fc3-eee2-49d7-9db4-1b61fb683351 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:59.006462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2e914-db0e-4ce3-9eab-858a23cee6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:59.006670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2e914-db0e-4ce3-9eab-858a23cee6a3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:59.006984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2e914-db0e-4ce3-9eab-858a23cee6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:59.007468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2e914-db0e-4ce3-9eab-858a23cee6a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:59.008181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2e914-db0e-4ce3-9eab-858a23cee6a3 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=a6cc40a4154043ada48161d66116e3c8, user.id=fee2283327204c6f8ee811e2c566e379) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:59.008481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2e914-db0e-4ce3-9eab-858a23cee6a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:59.010152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2e914-db0e-4ce3-9eab-858a23cee6a3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:59.024202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 88/170] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:28:58 2022] GET /identity/v3/role_assignments?user.id=fee2283327204c6f8ee811e2c566e379&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:59.025058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2e914-db0e-4ce3-9eab-858a23cee6a3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:59.075303 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab91fc3-eee2-49d7-9db4-1b61fb683351 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:59.097576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:59.124093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb712d8-6482-4c1d-a6aa-ad8ff5de0382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:59.125197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb712d8-6482-4c1d-a6aa-ad8ff5de0382 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:59.126751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb712d8-6482-4c1d-a6aa-ad8ff5de0382 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:59.127239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb712d8-6482-4c1d-a6aa-ad8ff5de0382 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:59.143412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb712d8-6482-4c1d-a6aa-ad8ff5de0382 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2cb712d8-6482-4c1d-a6aa-ad8ff5de0382 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:28:59.152404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:28:59.153763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:28:59.154537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:28:59.158110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 83/171] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:28:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:28:59.169234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb712d8-6482-4c1d-a6aa-ad8ff5de0382 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:59.226055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:59.245140 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2e914-db0e-4ce3-9eab-858a23cee6a3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:59.294088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb712d8-6482-4c1d-a6aa-ad8ff5de0382 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:59.319256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc55089-00ba-49aa-a22f-0e7eef640c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:59.319915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc55089-00ba-49aa-a22f-0e7eef640c1f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:59.320197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc55089-00ba-49aa-a22f-0e7eef640c1f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:59.320472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc55089-00ba-49aa-a22f-0e7eef640c1f None admin] PATH_INFO: `/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:59.325684 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc55089-00ba-49aa-a22f-0e7eef640c1f None admin] RBAC: Authorizing `identity:get_user(user_id=0cac9a0d6d824acfbc13df63c8e5dff8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:59.327301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc55089-00ba-49aa-a22f-0e7eef640c1f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:59.328966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 89/172] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:59 2022] GET /identity/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8 => generated 275 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:59.341353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc55089-00ba-49aa-a22f-0e7eef640c1f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:59.343047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ef937-1f68-46ef-9107-cad2cfd046df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:59.343991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ef937-1f68-46ef-9107-cad2cfd046df None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:59.344468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ef937-1f68-46ef-9107-cad2cfd046df None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:59.344967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547ef937-1f68-46ef-9107-cad2cfd046df None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:59.345851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ef937-1f68-46ef-9107-cad2cfd046df None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:28:59.346617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ef937-1f68-46ef-9107-cad2cfd046df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:59.348480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547ef937-1f68-46ef-9107-cad2cfd046df None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:59.359842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 84/173] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:28:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:59.369852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ef937-1f68-46ef-9107-cad2cfd046df None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:59.459829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:59.598760 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc55089-00ba-49aa-a22f-0e7eef640c1f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:59.637284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547ef937-1f68-46ef-9107-cad2cfd046df None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:28:59.670853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82c3309-11b2-47ec-902a-2be60ab46dc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:59.671472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c3309-11b2-47ec-902a-2be60ab46dc7 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:59.671749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c3309-11b2-47ec-902a-2be60ab46dc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:59.672019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82c3309-11b2-47ec-902a-2be60ab46dc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:59.702719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:59.741848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ac811-570d-44a9-a2f6-e67a9aeb298b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:28:59.742505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ac811-570d-44a9-a2f6-e67a9aeb298b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:59.742784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ac811-570d-44a9-a2f6-e67a9aeb298b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:59.743059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ac811-570d-44a9-a2f6-e67a9aeb298b None admin] PATH_INFO: `/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:59.744251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ac811-570d-44a9-a2f6-e67a9aeb298b None admin] RBAC: Authorizing `identity:get_user(user_id=0cac9a0d6d824acfbc13df63c8e5dff8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:59.745725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830ac811-570d-44a9-a2f6-e67a9aeb298b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:59.746663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c3309-11b2-47ec-902a-2be60ab46dc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:28:59.747589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 85/174] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:28:59 2022] GET /identity/v3/users/0cac9a0d6d824acfbc13df63c8e5dff8 => generated 275 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:28:59.748132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82c3309-11b2-47ec-902a-2be60ab46dc7 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:28:59.757991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ac811-570d-44a9-a2f6-e67a9aeb298b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:59.861113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 90/175] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 22:28:59.863067 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e779721-0a6e-46fd-bb40-023da61459ee None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:28:59.863355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e779721-0a6e-46fd-bb40-023da61459ee None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:28:59.863837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e779721-0a6e-46fd-bb40-023da61459ee None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:28:59.866045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 91/176] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:28:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:28:59.869815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e779721-0a6e-46fd-bb40-023da61459ee None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:28:59.943671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:28:59.994540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e779721-0a6e-46fd-bb40-023da61459ee None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:00.039824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830ac811-570d-44a9-a2f6-e67a9aeb298b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:00.053084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:00.069150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b566bde-e639-4720-9d00-e8195baa6eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:00.072795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b566bde-e639-4720-9d00-e8195baa6eea None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.072795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b566bde-e639-4720-9d00-e8195baa6eea None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.072795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b566bde-e639-4720-9d00-e8195baa6eea None admin] PATH_INFO: `/v3/projects/929f910a2e0d4acc995c08172ce891f3` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.072795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b566bde-e639-4720-9d00-e8195baa6eea None admin] RBAC: Authorizing `identity:get_project(project_id=929f910a2e0d4acc995c08172ce891f3)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:00.073091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b566bde-e639-4720-9d00-e8195baa6eea None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:00.078388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 92/177] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:28:59 2022] GET /identity/v3/projects/929f910a2e0d4acc995c08172ce891f3 => generated 312 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:00.079705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641d1f-8914-499c-b47a-5af3635d729c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.079980 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641d1f-8914-499c-b47a-5af3635d729c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.080247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93641d1f-8914-499c-b47a-5af3635d729c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.108464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf877e5-7e78-4328-8c26-3feac95e7af1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:00.109113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf877e5-7e78-4328-8c26-3feac95e7af1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.109386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf877e5-7e78-4328-8c26-3feac95e7af1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.109651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf877e5-7e78-4328-8c26-3feac95e7af1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.113030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-93641d1f-8914-499c-b47a-5af3635d729c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:00.114165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-93641d1f-8914-499c-b47a-5af3635d729c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:00.187176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf877e5-7e78-4328-8c26-3feac95e7af1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:00.188599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf877e5-7e78-4328-8c26-3feac95e7af1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:00.191380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 93/178] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:00.192461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93641d1f-8914-499c-b47a-5af3635d729c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:00.249000 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:00.321720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93641d1f-8914-499c-b47a-5af3635d729c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:00.334249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 86/179] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:28:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 577 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:00.335697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8a707-234d-472a-b238-2a77a091e548 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.336085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8a707-234d-472a-b238-2a77a091e548 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.336462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b8a707-234d-472a-b238-2a77a091e548 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.343267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 87/180] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:00.343267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8a707-234d-472a-b238-2a77a091e548 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:00.361135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:00.392092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698b10c8-dc95-4565-8297-48e032f26089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:00.392662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b10c8-dc95-4565-8297-48e032f26089 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.392973 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b10c8-dc95-4565-8297-48e032f26089 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.393247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698b10c8-dc95-4565-8297-48e032f26089 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.393982 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b10c8-dc95-4565-8297-48e032f26089 None admin] RBAC: Adding query filter params (role.id=aac9ca64da09463ba9b3966c7188d7ee, scope.project.id=929f910a2e0d4acc995c08172ce891f3, user.id=0cac9a0d6d824acfbc13df63c8e5dff8) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:00.394290 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b10c8-dc95-4565-8297-48e032f26089 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:00.396318 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698b10c8-dc95-4565-8297-48e032f26089 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:00.411903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 94/181] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:00 2022] GET /identity/v3/role_assignments?user.id=0cac9a0d6d824acfbc13df63c8e5dff8&scope.project.id=929f910a2e0d4acc995c08172ce891f3&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:00.413394 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c847a2-8048-4e30-969b-720c4961177f None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.413679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c847a2-8048-4e30-969b-720c4961177f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.414209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c847a2-8048-4e30-969b-720c4961177f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.455120 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-a5c847a2-8048-4e30-969b-720c4961177f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:00.456351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a5c847a2-8048-4e30-969b-720c4961177f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:00.482150 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b8a707-234d-472a-b238-2a77a091e548 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:00.545578 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a93249a-b928-438b-a75b-ea04ec2d39ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:00.546222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93249a-b928-438b-a75b-ea04ec2d39ca None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.546508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93249a-b928-438b-a75b-ea04ec2d39ca None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.546784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a93249a-b928-438b-a75b-ea04ec2d39ca None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.547970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93249a-b928-438b-a75b-ea04ec2d39ca None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:00.552860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a93249a-b928-438b-a75b-ea04ec2d39ca None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:00.552860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 88/182] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:00 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:00.554194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2581-1393-4a53-b809-824fea80d53a None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.554632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2581-1393-4a53-b809-824fea80d53a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.555027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fd2581-1393-4a53-b809-824fea80d53a None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.556550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 89/183] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:00.557562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd2581-1393-4a53-b809-824fea80d53a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:00.595656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 95/184] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:00.596882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c847a2-8048-4e30-969b-720c4961177f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:00.666258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:00.667817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:00.695838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fd2581-1393-4a53-b809-824fea80d53a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:00.733589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c847a2-8048-4e30-969b-720c4961177f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:00.748750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82743a27-e263-4ce5-8f88-9b40c8538104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:00.749606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743a27-e263-4ce5-8f88-9b40c8538104 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.749725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743a27-e263-4ce5-8f88-9b40c8538104 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.749969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82743a27-e263-4ce5-8f88-9b40c8538104 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.761497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82743a27-e263-4ce5-8f88-9b40c8538104 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-82743a27-e263-4ce5-8f88-9b40c8538104 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:00.766309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:00.767168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:00.767938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 90/185] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:29:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:00.768569 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82afb-8268-45de-9c6b-dd3248ccf881 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.769126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82afb-8268-45de-9c6b-dd3248ccf881 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.769126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e82afb-8268-45de-9c6b-dd3248ccf881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.780897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b935a85a-195d-4249-8d3f-939756901dc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:00.781917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935a85a-195d-4249-8d3f-939756901dc4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.782198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935a85a-195d-4249-8d3f-939756901dc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.782464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b935a85a-195d-4249-8d3f-939756901dc4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.783338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935a85a-195d-4249-8d3f-939756901dc4 None admin] RBAC: Adding query filter params (role.id=952084593c184a78a952e1797a0b6c6b, scope.project.id=a6cc40a4154043ada48161d66116e3c8, user.id=0cac9a0d6d824acfbc13df63c8e5dff8) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:00.783650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935a85a-195d-4249-8d3f-939756901dc4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:00.785118 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b935a85a-195d-4249-8d3f-939756901dc4 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:00.787537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-24e82afb-8268-45de-9c6b-dd3248ccf881 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:00.788227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-24e82afb-8268-45de-9c6b-dd3248ccf881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:00.793013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 96/186] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:00 2022] GET /identity/v3/role_assignments?user.id=0cac9a0d6d824acfbc13df63c8e5dff8&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=952084593c184a78a952e1797a0b6c6b => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:00.794506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44e618-837d-49c1-b303-118a08f566a5 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.795075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44e618-837d-49c1-b303-118a08f566a5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.795595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44e618-837d-49c1-b303-118a08f566a5 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.797613 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 97/187] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:00.799086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44e618-837d-49c1-b303-118a08f566a5 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:00.863347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 91/188] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:00.869065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254c387-72c4-45ad-aa77-a47b06a2cb75 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.869065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254c387-72c4-45ad-aa77-a47b06a2cb75 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.869065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f254c387-72c4-45ad-aa77-a47b06a2cb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.900110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f254c387-72c4-45ad-aa77-a47b06a2cb75 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:00.901586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f254c387-72c4-45ad-aa77-a47b06a2cb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:00.955709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44e618-837d-49c1-b303-118a08f566a5 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:00.968053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:00.979733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:00.990901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 92/189] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:00.992115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254c387-72c4-45ad-aa77-a47b06a2cb75 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:00.996116 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde66134-6646-4bb3-85f6-d51d2b187103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:00.996724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde66134-6646-4bb3-85f6-d51d2b187103 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:00.997039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde66134-6646-4bb3-85f6-d51d2b187103 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:00.997306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde66134-6646-4bb3-85f6-d51d2b187103 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:00.997966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde66134-6646-4bb3-85f6-d51d2b187103 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:00.998259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde66134-6646-4bb3-85f6-d51d2b187103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:00.999614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde66134-6646-4bb3-85f6-d51d2b187103 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:01.009902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 98/190] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:01.016060 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde66134-6646-4bb3-85f6-d51d2b187103 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:01.115642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f254c387-72c4-45ad-aa77-a47b06a2cb75 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:01.127697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde66134-6646-4bb3-85f6-d51d2b187103 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:01.158176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72974fcf-a601-47cc-bbf0-491ac801285b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:01.158808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72974fcf-a601-47cc-bbf0-491ac801285b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:01.159083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72974fcf-a601-47cc-bbf0-491ac801285b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:01.159525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72974fcf-a601-47cc-bbf0-491ac801285b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:01.164985 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc426077-44b8-4af2-9d30-64eef23b6c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:01.166063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426077-44b8-4af2-9d30-64eef23b6c36 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:01.166486 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426077-44b8-4af2-9d30-64eef23b6c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:01.166896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc426077-44b8-4af2-9d30-64eef23b6c36 None admin] PATH_INFO: `/v3/users/9aeb4f850cff4f92ab5fe65a8665a622` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:01.172577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72974fcf-a601-47cc-bbf0-491ac801285b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-72974fcf-a601-47cc-bbf0-491ac801285b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:01.179263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:01.180343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:01.181217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 93/191] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:01.184538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72974fcf-a601-47cc-bbf0-491ac801285b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:01.187930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc426077-44b8-4af2-9d30-64eef23b6c36 None admin] RBAC: Authorizing `identity:get_user(user_id=9aeb4f850cff4f92ab5fe65a8665a622)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:01.190423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc426077-44b8-4af2-9d30-64eef23b6c36 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:01.192351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 99/192] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:01 2022] GET /identity/v3/users/9aeb4f850cff4f92ab5fe65a8665a622 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:01.193164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc426077-44b8-4af2-9d30-64eef23b6c36 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:01.237093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:01.257743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:01.307090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72974fcf-a601-47cc-bbf0-491ac801285b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:01.311365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc426077-44b8-4af2-9d30-64eef23b6c36 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:01.341727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd647d3d-d71c-4bc0-8a6c-e0e24879b62c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:01.342352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd647d3d-d71c-4bc0-8a6c-e0e24879b62c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:01.342628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd647d3d-d71c-4bc0-8a6c-e0e24879b62c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:01.342898 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd647d3d-d71c-4bc0-8a6c-e0e24879b62c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:01.345097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90e993-52de-4494-b94c-d4353a94899f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:01.345698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90e993-52de-4494-b94c-d4353a94899f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:01.345970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90e993-52de-4494-b94c-d4353a94899f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:01.346237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f90e993-52de-4494-b94c-d4353a94899f None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:01.346886 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90e993-52de-4494-b94c-d4353a94899f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:01.347182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90e993-52de-4494-b94c-d4353a94899f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:01.348536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f90e993-52de-4494-b94c-d4353a94899f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:01.352729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd647d3d-d71c-4bc0-8a6c-e0e24879b62c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:01.356531 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 100/193] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:01.357325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90e993-52de-4494-b94c-d4353a94899f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-cd647d3d-d71c-4bc0-8a6c-e0e24879b62c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:01.357562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:01.358406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:01.359176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 94/194] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:29:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:01.359831 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd647d3d-d71c-4bc0-8a6c-e0e24879b62c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:01.454569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:01.463485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd647d3d-d71c-4bc0-8a6c-e0e24879b62c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:01.463734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f90e993-52de-4494-b94c-d4353a94899f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:01.478679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:01.497951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b686baf-1aa6-403a-9c2c-d0be85069b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:01.498483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b686baf-1aa6-403a-9c2c-d0be85069b15 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:01.498754 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b686baf-1aa6-403a-9c2c-d0be85069b15 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:01.499016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b686baf-1aa6-403a-9c2c-d0be85069b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:01.499291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed38c2d-351f-4990-a95a-0a6759898334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:01.499925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed38c2d-351f-4990-a95a-0a6759898334 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:01.500202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed38c2d-351f-4990-a95a-0a6759898334 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:01.500473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed38c2d-351f-4990-a95a-0a6759898334 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:01.501612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed38c2d-351f-4990-a95a-0a6759898334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:01.503060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed38c2d-351f-4990-a95a-0a6759898334 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:01.504612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 95/195] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:01.509616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed38c2d-351f-4990-a95a-0a6759898334 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:01.566946 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b686baf-1aa6-403a-9c2c-d0be85069b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:01.568522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b686baf-1aa6-403a-9c2c-d0be85069b15 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:01.608050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed38c2d-351f-4990-a95a-0a6759898334 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:01.642212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e208a-7291-4af2-b165-93853f4dc945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:01.642915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e208a-7291-4af2-b165-93853f4dc945 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:01.643195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e208a-7291-4af2-b165-93853f4dc945 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:01.643467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1e208a-7291-4af2-b165-93853f4dc945 None admin] PATH_INFO: `/v3/users/2ab0da9f2e6f43dc9337b9fc7bc13c2a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:01.644596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e208a-7291-4af2-b165-93853f4dc945 None admin] RBAC: Authorizing `identity:get_user(user_id=2ab0da9f2e6f43dc9337b9fc7bc13c2a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:01.646057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1e208a-7291-4af2-b165-93853f4dc945 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:01.647793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 96/196] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:01 2022] GET /identity/v3/users/2ab0da9f2e6f43dc9337b9fc7bc13c2a => generated 289 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:01.648379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e208a-7291-4af2-b165-93853f4dc945 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:01.659704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:01.670044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 101/197] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:01.670826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b686baf-1aa6-403a-9c2c-d0be85069b15 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:01.697226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:01.751506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1e208a-7291-4af2-b165-93853f4dc945 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:01.777100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b686baf-1aa6-403a-9c2c-d0be85069b15 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:01.786912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715940b3-269e-48cd-9317-b8898e1e699e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:01.787518 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715940b3-269e-48cd-9317-b8898e1e699e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:01.787792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715940b3-269e-48cd-9317-b8898e1e699e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:01.788060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715940b3-269e-48cd-9317-b8898e1e699e None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:01.788711 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715940b3-269e-48cd-9317-b8898e1e699e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:01.789037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715940b3-269e-48cd-9317-b8898e1e699e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:01.790427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715940b3-269e-48cd-9317-b8898e1e699e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:01.798051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 97/198] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:01.798824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715940b3-269e-48cd-9317-b8898e1e699e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:01.810782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc925e-8878-4ccc-a1c7-2ccfbe51c3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:01.811388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc925e-8878-4ccc-a1c7-2ccfbe51c3be None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:01.811663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc925e-8878-4ccc-a1c7-2ccfbe51c3be None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:01.811926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc925e-8878-4ccc-a1c7-2ccfbe51c3be None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:01.812570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc925e-8878-4ccc-a1c7-2ccfbe51c3be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:01.814050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc925e-8878-4ccc-a1c7-2ccfbe51c3be None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:01.834177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 102/199] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:29:01 2022] POST /identity/v3/users => generated 297 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:29:01.835146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc925e-8878-4ccc-a1c7-2ccfbe51c3be None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:01.900353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:01.908387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715940b3-269e-48cd-9317-b8898e1e699e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:01.923412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:01.946435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bc154-3ed2-4c9a-9026-37c27ee7feef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:01.947057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bc154-3ed2-4c9a-9026-37c27ee7feef None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:01.947336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bc154-3ed2-4c9a-9026-37c27ee7feef None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:01.947605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03bc154-3ed2-4c9a-9026-37c27ee7feef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:01.969046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc925e-8878-4ccc-a1c7-2ccfbe51c3be None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:02.009061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e20f5a-332d-40bf-a451-02996d5edf5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:02.009679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e20f5a-332d-40bf-a451-02996d5edf5d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:02.009955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e20f5a-332d-40bf-a451-02996d5edf5d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:02.010228 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e20f5a-332d-40bf-a451-02996d5edf5d None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:02.011428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e20f5a-332d-40bf-a451-02996d5edf5d None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:02.012857 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e20f5a-332d-40bf-a451-02996d5edf5d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:02.014710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 103/200] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:01 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:02.019577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e20f5a-332d-40bf-a451-02996d5edf5d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:02.029602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bc154-3ed2-4c9a-9026-37c27ee7feef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:02.031094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03bc154-3ed2-4c9a-9026-37c27ee7feef None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:02.153626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e20f5a-332d-40bf-a451-02996d5edf5d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:02.154271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 98/201] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:02.155115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bc154-3ed2-4c9a-9026-37c27ee7feef None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:02.173487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:02.178375 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:02.201056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59678d1-704e-4051-9651-a290a6702337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:02.201696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59678d1-704e-4051-9651-a290a6702337 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:02.201975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59678d1-704e-4051-9651-a290a6702337 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:02.202250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59678d1-704e-4051-9651-a290a6702337 None admin] PATH_INFO: `/v3/users/9aeb4f850cff4f92ab5fe65a8665a622` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:02.203839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59678d1-704e-4051-9651-a290a6702337 None admin] RBAC: Authorizing `identity:get_user(user_id=9aeb4f850cff4f92ab5fe65a8665a622)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:02.205313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59678d1-704e-4051-9651-a290a6702337 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:02.207264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 104/202] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:02 2022] GET /identity/v3/users/9aeb4f850cff4f92ab5fe65a8665a622 => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:02.207953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59678d1-704e-4051-9651-a290a6702337 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:02.281600 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03bc154-3ed2-4c9a-9026-37c27ee7feef None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:02.321198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59678d1-704e-4051-9651-a290a6702337 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:02.325147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc020be-a34d-4e77-91d3-e504f67eb868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:02.325930 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc020be-a34d-4e77-91d3-e504f67eb868 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:02.326366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc020be-a34d-4e77-91d3-e504f67eb868 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:02.326771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc020be-a34d-4e77-91d3-e504f67eb868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:02.327729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc020be-a34d-4e77-91d3-e504f67eb868 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=ef03f831e4b94427a2be5482f556af22, user.id=9aeb4f850cff4f92ab5fe65a8665a622) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:02.328171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc020be-a34d-4e77-91d3-e504f67eb868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:02.330306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc020be-a34d-4e77-91d3-e504f67eb868 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:02.339255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 99/203] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:02 2022] GET /identity/v3/role_assignments?user.id=9aeb4f850cff4f92ab5fe65a8665a622&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:02.341046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc020be-a34d-4e77-91d3-e504f67eb868 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:02.364371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02624e01-ea41-436e-8b60-d43d1371833b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:02.365478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02624e01-ea41-436e-8b60-d43d1371833b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:02.365891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02624e01-ea41-436e-8b60-d43d1371833b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:02.366300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02624e01-ea41-436e-8b60-d43d1371833b None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:02.367808 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02624e01-ea41-436e-8b60-d43d1371833b None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:02.371178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02624e01-ea41-436e-8b60-d43d1371833b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:02.374379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 105/204] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:02 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:02.383630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02624e01-ea41-436e-8b60-d43d1371833b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:02.485607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:02.498291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:02.529186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc020be-a34d-4e77-91d3-e504f67eb868 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:02.601010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02624e01-ea41-436e-8b60-d43d1371833b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:02.605018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d3e8e-ec2f-4535-8df8-c9f02a7e254b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:02.605632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d3e8e-ec2f-4535-8df8-c9f02a7e254b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:02.605905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d3e8e-ec2f-4535-8df8-c9f02a7e254b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:02.606172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480d3e8e-ec2f-4535-8df8-c9f02a7e254b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:02.671579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9253b9e-19ed-437e-9812-e766b47eb368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:02.672985 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9253b9e-19ed-437e-9812-e766b47eb368 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:02.673256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9253b9e-19ed-437e-9812-e766b47eb368 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:02.673520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9253b9e-19ed-437e-9812-e766b47eb368 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:02.674247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9253b9e-19ed-437e-9812-e766b47eb368 None admin] RBAC: Adding query filter params (role.id=4f87bf3cce2d4008a95c1da7369d74bf, scope.project.id=a6cc40a4154043ada48161d66116e3c8, user.id=2ab0da9f2e6f43dc9337b9fc7bc13c2a) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:02.674549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9253b9e-19ed-437e-9812-e766b47eb368 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:02.675977 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9253b9e-19ed-437e-9812-e766b47eb368 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:02.697689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 106/205] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:02 2022] GET /identity/v3/role_assignments?user.id=2ab0da9f2e6f43dc9337b9fc7bc13c2a&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=4f87bf3cce2d4008a95c1da7369d74bf => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:02.741933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d3e8e-ec2f-4535-8df8-c9f02a7e254b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:02.743452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480d3e8e-ec2f-4535-8df8-c9f02a7e254b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:02.860314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:02.896874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 100/206] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:02.902828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9253b9e-19ed-437e-9812-e766b47eb368 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:03.012877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9253b9e-19ed-437e-9812-e766b47eb368 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:03.032463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:03.055342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42744d3f-b894-4e20-b641-c947ab578efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:03.055981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42744d3f-b894-4e20-b641-c947ab578efb None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:03.056256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42744d3f-b894-4e20-b641-c947ab578efb None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:03.056525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42744d3f-b894-4e20-b641-c947ab578efb None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:03.057691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42744d3f-b894-4e20-b641-c947ab578efb None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:03.059133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42744d3f-b894-4e20-b641-c947ab578efb None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:03.061096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 107/207] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:02 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:03.066864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d3e8e-ec2f-4535-8df8-c9f02a7e254b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:03.178309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480d3e8e-ec2f-4535-8df8-c9f02a7e254b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:03.219218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d086ca-15b9-4a3c-bd69-dc356b4abf98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:03.219826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d086ca-15b9-4a3c-bd69-dc356b4abf98 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:03.220095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d086ca-15b9-4a3c-bd69-dc356b4abf98 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:03.220358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d086ca-15b9-4a3c-bd69-dc356b4abf98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:03.221111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d086ca-15b9-4a3c-bd69-dc356b4abf98 None admin] RBAC: Adding query filter params (role.id=952084593c184a78a952e1797a0b6c6b, scope.project.id=ef03f831e4b94427a2be5482f556af22, user.id=9aeb4f850cff4f92ab5fe65a8665a622) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:03.221413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d086ca-15b9-4a3c-bd69-dc356b4abf98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:03.222843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d086ca-15b9-4a3c-bd69-dc356b4abf98 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:03.229321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 101/208] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:03 2022] GET /identity/v3/role_assignments?user.id=9aeb4f850cff4f92ab5fe65a8665a622&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=952084593c184a78a952e1797a0b6c6b => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:04.356054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a5b85-e39e-4156-b37a-384778365dbf None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:04.357090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a5b85-e39e-4156-b37a-384778365dbf None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:04.357569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45a5b85-e39e-4156-b37a-384778365dbf None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:04.359333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 108/209] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:04.365446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2205a41-c7c2-4580-b2d5-db1d7fe4546a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:04.366444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2205a41-c7c2-4580-b2d5-db1d7fe4546a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:04.366930 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2205a41-c7c2-4580-b2d5-db1d7fe4546a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:04.389580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f2205a41-c7c2-4580-b2d5-db1d7fe4546a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:04.390770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f2205a41-c7c2-4580-b2d5-db1d7fe4546a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:04.422362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31079f6c-f41b-424a-852d-c916ef5912bb None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:04.422717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31079f6c-f41b-424a-852d-c916ef5912bb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:04.423064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31079f6c-f41b-424a-852d-c916ef5912bb None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:04.424469 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 109/210] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:29:04.430245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569d1b7-3e97-4810-a215-5b2b2428c106 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:04.430523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569d1b7-3e97-4810-a215-5b2b2428c106 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:04.430793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d569d1b7-3e97-4810-a215-5b2b2428c106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:04.439833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:04.449810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d569d1b7-3e97-4810-a215-5b2b2428c106 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:04.451284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d569d1b7-3e97-4810-a215-5b2b2428c106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:04.484105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 102/211] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:04.528242 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 110/212] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:04.766160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2205a41-c7c2-4580-b2d5-db1d7fe4546a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:04.800246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569d1b7-3e97-4810-a215-5b2b2428c106 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:04.888020 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2205a41-c7c2-4580-b2d5-db1d7fe4546a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:04.906015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d569d1b7-3e97-4810-a215-5b2b2428c106 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:04.923267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:04.930800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9a86d-0451-4725-bb95-5b986477e4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:04.931943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9a86d-0451-4725-bb95-5b986477e4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:04.932381 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9a86d-0451-4725-bb95-5b986477e4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:04.932971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad9a86d-0451-4725-bb95-5b986477e4c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:04.943827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040a871b-caa6-4955-901c-40f7a3abecdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:04.944458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a871b-caa6-4955-901c-40f7a3abecdf None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:04.944734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a871b-caa6-4955-901c-40f7a3abecdf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:04.945021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040a871b-caa6-4955-901c-40f7a3abecdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:04.945684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad9a86d-0451-4725-bb95-5b986477e4c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:04.946130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a871b-caa6-4955-901c-40f7a3abecdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:04.947567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040a871b-caa6-4955-901c-40f7a3abecdf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:04.949326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 111/213] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-dad9a86d-0451-4725-bb95-5b986477e4c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:04.950766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:04.951616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 103/214] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:04.957444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9a86d-0451-4725-bb95-5b986477e4c6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:04.958742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1491e06-a65f-4af9-8502-81ca4e3fc56d None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:04.958742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1491e06-a65f-4af9-8502-81ca4e3fc56d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:04.958742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1491e06-a65f-4af9-8502-81ca4e3fc56d None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:04.958742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 112/215] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:04 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:04.962734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1491e06-a65f-4af9-8502-81ca4e3fc56d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:04.989278 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:05.067226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad9a86d-0451-4725-bb95-5b986477e4c6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:05.078497 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1491e06-a65f-4af9-8502-81ca4e3fc56d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:05.105107 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075368ae-457c-4d9a-81a2-bf6e1af6cd49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:05.105710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075368ae-457c-4d9a-81a2-bf6e1af6cd49 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:05.106124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075368ae-457c-4d9a-81a2-bf6e1af6cd49 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:05.106362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075368ae-457c-4d9a-81a2-bf6e1af6cd49 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:05.107121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075368ae-457c-4d9a-81a2-bf6e1af6cd49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:05.108557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075368ae-457c-4d9a-81a2-bf6e1af6cd49 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:05.116919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e08c4f2-5f13-4f54-b609-c9aee97fe105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:05.117527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08c4f2-5f13-4f54-b609-c9aee97fe105 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:05.117804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08c4f2-5f13-4f54-b609-c9aee97fe105 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:05.118123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e08c4f2-5f13-4f54-b609-c9aee97fe105 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:05.118794 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08c4f2-5f13-4f54-b609-c9aee97fe105 None admin] RBAC: Adding query filter params (name=member) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:05.119101 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08c4f2-5f13-4f54-b609-c9aee97fe105 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:05.120451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e08c4f2-5f13-4f54-b609-c9aee97fe105 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:05.129326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 113/216] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:05.130530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1490dd-4986-49c3-8d87-8b1659ff4165 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:05.130970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1490dd-4986-49c3-8d87-8b1659ff4165 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:05.131243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1490dd-4986-49c3-8d87-8b1659ff4165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:05.144704 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 104/217] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:29:04 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:29:05.146177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075368ae-457c-4d9a-81a2-bf6e1af6cd49 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:05.155109 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-1d1490dd-4986-49c3-8d87-8b1659ff4165 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:05.156432 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1d1490dd-4986-49c3-8d87-8b1659ff4165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:05.234878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:05.301209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:05.301441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 114/218] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:05.317432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075368ae-457c-4d9a-81a2-bf6e1af6cd49 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:05.359014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f97e47-1c22-4be9-ad0e-954dbb272ad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:05.359947 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f97e47-1c22-4be9-ad0e-954dbb272ad3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:05.360386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f97e47-1c22-4be9-ad0e-954dbb272ad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:05.360997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f97e47-1c22-4be9-ad0e-954dbb272ad3 None admin] PATH_INFO: `/v3/users/4420a089cd6740dc809c1e9ce9583b38` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:05.377094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f97e47-1c22-4be9-ad0e-954dbb272ad3 None admin] RBAC: Authorizing `identity:get_user(user_id=4420a089cd6740dc809c1e9ce9583b38)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:05.379305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f97e47-1c22-4be9-ad0e-954dbb272ad3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:05.381715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 105/219] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:05 2022] GET /identity/v3/users/4420a089cd6740dc809c1e9ce9583b38 => generated 297 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:05.391717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1490dd-4986-49c3-8d87-8b1659ff4165 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:05.429148 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ed3ba-2e83-4ba1-9bad-b82e1e10292c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:05.429532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ed3ba-2e83-4ba1-9bad-b82e1e10292c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:05.429917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3ed3ba-2e83-4ba1-9bad-b82e1e10292c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:05.433771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 106/220] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:05.440552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef395a39-4448-43f4-a75d-4bf696d0aab0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:05.440936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef395a39-4448-43f4-a75d-4bf696d0aab0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:05.441112 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef395a39-4448-43f4-a75d-4bf696d0aab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:05.464931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ef395a39-4448-43f4-a75d-4bf696d0aab0 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:05.466116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ef395a39-4448-43f4-a75d-4bf696d0aab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:05.511934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1490dd-4986-49c3-8d87-8b1659ff4165 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:05.553255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0f84b-1c9e-45ce-a2ac-5155f6020b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:05.553878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0f84b-1c9e-45ce-a2ac-5155f6020b5d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:05.554165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0f84b-1c9e-45ce-a2ac-5155f6020b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:05.554425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b0f84b-1c9e-45ce-a2ac-5155f6020b5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:05.561038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 107/221] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:05.626155 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:05.649905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef395a39-4448-43f4-a75d-4bf696d0aab0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:05.662085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0f84b-1c9e-45ce-a2ac-5155f6020b5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:05.664512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b0f84b-1c9e-45ce-a2ac-5155f6020b5d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:05.785131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:05.811519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 115/222] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:05.813677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef395a39-4448-43f4-a75d-4bf696d0aab0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:05.816801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0f84b-1c9e-45ce-a2ac-5155f6020b5d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:05.875319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41836fb-3a3d-49d9-9770-59349ad5f97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:05.875965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41836fb-3a3d-49d9-9770-59349ad5f97c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:05.876305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41836fb-3a3d-49d9-9770-59349ad5f97c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:05.876587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41836fb-3a3d-49d9-9770-59349ad5f97c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:05.890766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41836fb-3a3d-49d9-9770-59349ad5f97c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-c41836fb-3a3d-49d9-9770-59349ad5f97c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:05.898045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:05.899139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:05.900117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 108/223] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:29:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:05.901542 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41836fb-3a3d-49d9-9770-59349ad5f97c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:05.925057 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:05.954705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b0f84b-1c9e-45ce-a2ac-5155f6020b5d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:05.991108 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479981b3-4e73-4bc0-a879-85c457e4fe68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:05.992128 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479981b3-4e73-4bc0-a879-85c457e4fe68 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:05.992626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479981b3-4e73-4bc0-a879-85c457e4fe68 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:05.993123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479981b3-4e73-4bc0-a879-85c457e4fe68 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:05.994459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479981b3-4e73-4bc0-a879-85c457e4fe68 None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:05.996084 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479981b3-4e73-4bc0-a879-85c457e4fe68 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:05.997952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 116/224] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:05 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:05.999155 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479981b3-4e73-4bc0-a879-85c457e4fe68 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:06.038878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41836fb-3a3d-49d9-9770-59349ad5f97c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:06.085321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d997b-c3a3-428c-9098-dac405a85f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:06.086754 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d997b-c3a3-428c-9098-dac405a85f2e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:06.087500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d997b-c3a3-428c-9098-dac405a85f2e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:06.087964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028d997b-c3a3-428c-9098-dac405a85f2e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:06.091000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:06.103670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028d997b-c3a3-428c-9098-dac405a85f2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:06.110722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479981b3-4e73-4bc0-a879-85c457e4fe68 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-028d997b-c3a3-428c-9098-dac405a85f2e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:06.111214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:06.112066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:06.112897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:06.114060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 109/225] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:29:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:06.120061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d997b-c3a3-428c-9098-dac405a85f2e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:06.154725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9901231-12f7-4710-b7a3-e849b03a6f5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:06.155685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9901231-12f7-4710-b7a3-e849b03a6f5a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:06.156164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9901231-12f7-4710-b7a3-e849b03a6f5a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:06.156622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9901231-12f7-4710-b7a3-e849b03a6f5a None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:06.157555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9901231-12f7-4710-b7a3-e849b03a6f5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:06.158046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9901231-12f7-4710-b7a3-e849b03a6f5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:06.159630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9901231-12f7-4710-b7a3-e849b03a6f5a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:06.169183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 117/226] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:06.170252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9901231-12f7-4710-b7a3-e849b03a6f5a None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:06.174657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:06.253027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028d997b-c3a3-428c-9098-dac405a85f2e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:06.287720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9901231-12f7-4710-b7a3-e849b03a6f5a None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:06.299671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324a38c-dd75-490f-8615-3789a3c430cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:06.300409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a38c-dd75-490f-8615-3789a3c430cd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:06.302175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a38c-dd75-490f-8615-3789a3c430cd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:06.302678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0324a38c-dd75-490f-8615-3789a3c430cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:06.303417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324a38c-dd75-490f-8615-3789a3c430cd None admin] RBAC: Adding query filter params (role.id=aac9ca64da09463ba9b3966c7188d7ee, scope.project.id=ef03f831e4b94427a2be5482f556af22, user.id=4420a089cd6740dc809c1e9ce9583b38) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:06.303726 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324a38c-dd75-490f-8615-3789a3c430cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:06.305226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0324a38c-dd75-490f-8615-3789a3c430cd None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:06.316878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 110/227] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:06 2022] GET /identity/v3/role_assignments?user.id=4420a089cd6740dc809c1e9ce9583b38&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:06.316878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324a38c-dd75-490f-8615-3789a3c430cd None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:06.356375 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8108d6-f046-4567-949b-797f95d49085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:06.357368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8108d6-f046-4567-949b-797f95d49085 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:06.357801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8108d6-f046-4567-949b-797f95d49085 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:06.358212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8108d6-f046-4567-949b-797f95d49085 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:06.359035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8108d6-f046-4567-949b-797f95d49085 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:06.359477 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8108d6-f046-4567-949b-797f95d49085 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:06.361125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8108d6-f046-4567-949b-797f95d49085 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:06.384141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 118/228] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:06.385395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8108d6-f046-4567-949b-797f95d49085 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:06.444344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:06.487246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:06.535249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8108d6-f046-4567-949b-797f95d49085 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:06.540712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0324a38c-dd75-490f-8615-3789a3c430cd None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:06.600654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e6b16-b26f-461f-9e30-a9db5028baec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:06.601322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6b16-b26f-461f-9e30-a9db5028baec None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:06.601593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6b16-b26f-461f-9e30-a9db5028baec None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:06.601865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e6b16-b26f-461f-9e30-a9db5028baec None admin] PATH_INFO: `/v3/users/9aeb4f850cff4f92ab5fe65a8665a622` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:06.603001 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e6b16-b26f-461f-9e30-a9db5028baec None admin] RBAC: Authorizing `identity:get_user(user_id=9aeb4f850cff4f92ab5fe65a8665a622)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:06.604437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192e6b16-b26f-461f-9e30-a9db5028baec None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:06.607068 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 111/229] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:06 2022] GET /identity/v3/users/9aeb4f850cff4f92ab5fe65a8665a622 => generated 283 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:06.608613 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e6b3a-0297-45a3-9d99-3c8b4ad449b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:06.609292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e6b3a-0297-45a3-9d99-3c8b4ad449b4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:06.609621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e6b3a-0297-45a3-9d99-3c8b4ad449b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:06.609897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e6b3a-0297-45a3-9d99-3c8b4ad449b4 None admin] PATH_INFO: `/v3/users/9aeb4f850cff4f92ab5fe65a8665a622` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:06.611082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e6b3a-0297-45a3-9d99-3c8b4ad449b4 None admin] RBAC: Authorizing `identity:get_user(user_id=9aeb4f850cff4f92ab5fe65a8665a622)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:06.612566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e6b16-b26f-461f-9e30-a9db5028baec None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:06.612832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e6b3a-0297-45a3-9d99-3c8b4ad449b4 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:06.614478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 119/230] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:06 2022] GET /identity/v3/users/9aeb4f850cff4f92ab5fe65a8665a622 => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:06.621972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e6b3a-0297-45a3-9d99-3c8b4ad449b4 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:06.749494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e6b3a-0297-45a3-9d99-3c8b4ad449b4 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:06.793265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723ef36-5fab-461a-84bc-93d93a598268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:06.793903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723ef36-5fab-461a-84bc-93d93a598268 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:06.794180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723ef36-5fab-461a-84bc-93d93a598268 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:06.794452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6723ef36-5fab-461a-84bc-93d93a598268 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:06.797733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723ef36-5fab-461a-84bc-93d93a598268 None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:06.799203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6723ef36-5fab-461a-84bc-93d93a598268 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:06.800868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 120/231] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:06 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:06.808074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723ef36-5fab-461a-84bc-93d93a598268 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:06.814053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:06.881789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192e6b16-b26f-461f-9e30-a9db5028baec None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:06.943794 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:06.962056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0bd88-29d7-4074-8354-20fadda1b5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:06.962697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0bd88-29d7-4074-8354-20fadda1b5ce None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:06.962972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0bd88-29d7-4074-8354-20fadda1b5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:06.963242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd0bd88-29d7-4074-8354-20fadda1b5ce None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:06.969507 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0bd88-29d7-4074-8354-20fadda1b5ce None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:06.971089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd0bd88-29d7-4074-8354-20fadda1b5ce None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:06.972993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 112/232] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:06 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 361 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:06.979430 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0bd88-29d7-4074-8354-20fadda1b5ce None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:06.985599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6723ef36-5fab-461a-84bc-93d93a598268 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:07.021455 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a07d4-ac43-4039-a981-8cd72b30b24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:07.022563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a07d4-ac43-4039-a981-8cd72b30b24c None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:07.023034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a07d4-ac43-4039-a981-8cd72b30b24c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:07.023496 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a07d4-ac43-4039-a981-8cd72b30b24c None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22/users/9aeb4f850cff4f92ab5fe65a8665a622/roles/952084593c184a78a952e1797a0b6c6b` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:07.025865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a07d4-ac43-4039-a981-8cd72b30b24c None admin] RBAC: Authorizing `identity:create_grant(project_id=ef03f831e4b94427a2be5482f556af22, user_id=9aeb4f850cff4f92ab5fe65a8665a622, role_id=952084593c184a78a952e1797a0b6c6b)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:07.027508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a07d4-ac43-4039-a981-8cd72b30b24c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:07.046905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 121/233] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:29:06 2022] PUT /identity/v3/projects/ef03f831e4b94427a2be5482f556af22/users/9aeb4f850cff4f92ab5fe65a8665a622/roles/952084593c184a78a952e1797a0b6c6b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:07.148544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd0bd88-29d7-4074-8354-20fadda1b5ce None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:07.191083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c19825c-49aa-4d32-8248-205394bb8724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:07.192107 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19825c-49aa-4d32-8248-205394bb8724 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:07.193214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19825c-49aa-4d32-8248-205394bb8724 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:07.193780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c19825c-49aa-4d32-8248-205394bb8724 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22/users/9aeb4f850cff4f92ab5fe65a8665a622/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:07.195898 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19825c-49aa-4d32-8248-205394bb8724 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef03f831e4b94427a2be5482f556af22, user_id=9aeb4f850cff4f92ab5fe65a8665a622, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:07.197602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c19825c-49aa-4d32-8248-205394bb8724 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:07.231859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 113/234] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:29:06 2022] PUT /identity/v3/projects/ef03f831e4b94427a2be5482f556af22/users/9aeb4f850cff4f92ab5fe65a8665a622/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:29:07.997210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60635a6f-693f-4814-b68d-c50af962d9a9 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:07.997721 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60635a6f-693f-4814-b68d-c50af962d9a9 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:07.997794 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60635a6f-693f-4814-b68d-c50af962d9a9 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:07.999364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 122/235] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:08.004919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002763a-146d-4f58-87d5-2c6adc45952b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:08.005949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002763a-146d-4f58-87d5-2c6adc45952b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:08.006279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e002763a-146d-4f58-87d5-2c6adc45952b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:08.032738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e002763a-146d-4f58-87d5-2c6adc45952b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:08.033986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e002763a-146d-4f58-87d5-2c6adc45952b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:08.051287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e68a-0b53-4ae2-a0fe-b3179dce09b3 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:08.051607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e68a-0b53-4ae2-a0fe-b3179dce09b3 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:08.051868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4242e68a-0b53-4ae2-a0fe-b3179dce09b3 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:08.054147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 123/236] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:08.059394 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5566c7c-1404-47b1-b064-5961082b0434 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:08.059873 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5566c7c-1404-47b1-b064-5961082b0434 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:08.059997 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5566c7c-1404-47b1-b064-5961082b0434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:08.079277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d5566c7c-1404-47b1-b064-5961082b0434 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:08.080469 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d5566c7c-1404-47b1-b064-5961082b0434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:08.107082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:08.128545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 114/237] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:08.170988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:08.187277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 124/238] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:08.475059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002763a-146d-4f58-87d5-2c6adc45952b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:08.495240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5566c7c-1404-47b1-b064-5961082b0434 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:08.634805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e002763a-146d-4f58-87d5-2c6adc45952b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:08.673117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97541e3-d422-4331-8702-f4c567e81bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:08.674089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97541e3-d422-4331-8702-f4c567e81bc8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:08.674557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97541e3-d422-4331-8702-f4c567e81bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:08.675014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97541e3-d422-4331-8702-f4c567e81bc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:08.676413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97541e3-d422-4331-8702-f4c567e81bc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:08.678712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97541e3-d422-4331-8702-f4c567e81bc8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:08.680504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 115/239] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:08.684943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5566c7c-1404-47b1-b064-5961082b0434 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:08.688796 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97541e3-d422-4331-8702-f4c567e81bc8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:08.722041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e212a-6288-4f78-add1-3f554906f4ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:08.723042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e212a-6288-4f78-add1-3f554906f4ef None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:08.723517 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e212a-6288-4f78-add1-3f554906f4ef None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:08.724042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944e212a-6288-4f78-add1-3f554906f4ef None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:08.744848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944e212a-6288-4f78-add1-3f554906f4ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-944e212a-6288-4f78-add1-3f554906f4ef None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:08.753259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:08.754186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:08.755644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 125/240] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:08.770982 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e212a-6288-4f78-add1-3f554906f4ef None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:08.801160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:08.872495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97541e3-d422-4331-8702-f4c567e81bc8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:08.881239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:08.950476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9565e29-7dea-485e-a010-1959ee3d7c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:08.951084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9565e29-7dea-485e-a010-1959ee3d7c65 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:08.951356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9565e29-7dea-485e-a010-1959ee3d7c65 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:08.951621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9565e29-7dea-485e-a010-1959ee3d7c65 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:08.952266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9565e29-7dea-485e-a010-1959ee3d7c65 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:08.953733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9565e29-7dea-485e-a010-1959ee3d7c65 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:08.967548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944e212a-6288-4f78-add1-3f554906f4ef None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:08.997382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 116/241] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:29:08 2022] POST /identity/v3/users => generated 293 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:29:09.013546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90eabb8-fb12-4e1d-88e2-0088d35087b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:09.014207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90eabb8-fb12-4e1d-88e2-0088d35087b3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:09.014446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90eabb8-fb12-4e1d-88e2-0088d35087b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:09.014716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90eabb8-fb12-4e1d-88e2-0088d35087b3 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:09.015470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90eabb8-fb12-4e1d-88e2-0088d35087b3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:09.015767 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90eabb8-fb12-4e1d-88e2-0088d35087b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:09.017318 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90eabb8-fb12-4e1d-88e2-0088d35087b3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:09.034619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 126/242] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:09.041572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9565e29-7dea-485e-a010-1959ee3d7c65 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:09.178185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9565e29-7dea-485e-a010-1959ee3d7c65 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:09.197156 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:09.211625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552971f-6e0c-404b-9e5f-81b3dd231e71 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:09.211926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552971f-6e0c-404b-9e5f-81b3dd231e71 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:09.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b552971f-6e0c-404b-9e5f-81b3dd231e71 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:09.213707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 127/243] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:09.222336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0afc94-d219-465b-a715-c773c4e25b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:09.223204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0afc94-d219-465b-a715-c773c4e25b7d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:09.223486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0afc94-d219-465b-a715-c773c4e25b7d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:09.224426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0afc94-d219-465b-a715-c773c4e25b7d None admin] PATH_INFO: `/v3/users/c9a69256fd19439a8c5ce59f32871fb6` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:09.228189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b2a85-a9c3-4060-83ae-4233c6d2a61f None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:09.228466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b2a85-a9c3-4060-83ae-4233c6d2a61f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:09.228738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b2a85-a9c3-4060-83ae-4233c6d2a61f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:09.255435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0afc94-d219-465b-a715-c773c4e25b7d None admin] RBAC: Authorizing `identity:get_user(user_id=c9a69256fd19439a8c5ce59f32871fb6)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:09.255788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-871b2a85-a9c3-4060-83ae-4233c6d2a61f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:09.257403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-871b2a85-a9c3-4060-83ae-4233c6d2a61f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:09.257908 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0afc94-d219-465b-a715-c773c4e25b7d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:09.259727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 117/244] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:09 2022] GET /identity/v3/users/c9a69256fd19439a8c5ce59f32871fb6 => generated 297 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:09.266447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0afc94-d219-465b-a715-c773c4e25b7d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:09.350225 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 128/245] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 06 22:29:09.397497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0afc94-d219-465b-a715-c773c4e25b7d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:09.435682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabe714-d988-4b5d-84c2-19d68effac68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:09.436293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabe714-d988-4b5d-84c2-19d68effac68 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:09.436612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabe714-d988-4b5d-84c2-19d68effac68 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:09.436922 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eabe714-d988-4b5d-84c2-19d68effac68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:09.475921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:09.514688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eabe714-d988-4b5d-84c2-19d68effac68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:09.516082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eabe714-d988-4b5d-84c2-19d68effac68 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:09.637248 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 118/246] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 22:29:09.642544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b2a85-a9c3-4060-83ae-4233c6d2a61f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:09.674000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabe714-d988-4b5d-84c2-19d68effac68 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:09.709641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:09.768967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:09.778699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b2a85-a9c3-4060-83ae-4233c6d2a61f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:09.816736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eabe714-d988-4b5d-84c2-19d68effac68 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:09.832272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52467668-cd1b-445b-9944-bb6438a5a520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:09.833013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52467668-cd1b-445b-9944-bb6438a5a520 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:09.833864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52467668-cd1b-445b-9944-bb6438a5a520 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:09.834232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52467668-cd1b-445b-9944-bb6438a5a520 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:09.835400 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52467668-cd1b-445b-9944-bb6438a5a520 None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:09.836880 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52467668-cd1b-445b-9944-bb6438a5a520 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:09.838843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 129/247] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:09 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:09.840002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcc666-0571-4626-87ed-8616c6a4c121 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:09.840281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcc666-0571-4626-87ed-8616c6a4c121 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:09.840553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdcc666-0571-4626-87ed-8616c6a4c121 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:09.842952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 130/248] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:09.847529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcc666-0571-4626-87ed-8616c6a4c121 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:09.865402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f650c551-fb92-462e-a1e1-43885597dad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:09.866631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650c551-fb92-462e-a1e1-43885597dad1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:09.867577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650c551-fb92-462e-a1e1-43885597dad1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:09.868309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f650c551-fb92-462e-a1e1-43885597dad1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:09.885079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f650c551-fb92-462e-a1e1-43885597dad1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f650c551-fb92-462e-a1e1-43885597dad1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:09.893471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:09.894444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:09.895302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:09.898360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 119/249] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:09.900719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e364f68-99a9-4f22-b9a8-42a8644e7c7d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:09.901306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e364f68-99a9-4f22-b9a8-42a8644e7c7d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:09.902429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e364f68-99a9-4f22-b9a8-42a8644e7c7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:09.931584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2e364f68-99a9-4f22-b9a8-42a8644e7c7d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:09.932816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2e364f68-99a9-4f22-b9a8-42a8644e7c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:09.977978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdcc666-0571-4626-87ed-8616c6a4c121 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:09.992249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:10.014754 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94944a-e077-4a32-90e8-8f962ad57faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:10.015363 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94944a-e077-4a32-90e8-8f962ad57faf None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:10.015622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94944a-e077-4a32-90e8-8f962ad57faf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:10.015893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f94944a-e077-4a32-90e8-8f962ad57faf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:10.016620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94944a-e077-4a32-90e8-8f962ad57faf None admin] RBAC: Adding query filter params (role.id=4f87bf3cce2d4008a95c1da7369d74bf, scope.project.id=ef03f831e4b94427a2be5482f556af22, user.id=c9a69256fd19439a8c5ce59f32871fb6) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:10.016966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94944a-e077-4a32-90e8-8f962ad57faf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:10.018663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f94944a-e077-4a32-90e8-8f962ad57faf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:10.026070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 131/250] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:09 2022] GET /identity/v3/role_assignments?user.id=c9a69256fd19439a8c5ce59f32871fb6&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=4f87bf3cce2d4008a95c1da7369d74bf => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:10.026954 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94944a-e077-4a32-90e8-8f962ad57faf None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:10.040195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 120/251] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:10.041492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33b447-85db-4414-9945-c9e30e6d40f5 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:10.043067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33b447-85db-4414-9945-c9e30e6d40f5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:10.043620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c33b447-85db-4414-9945-c9e30e6d40f5 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:10.044739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 121/252] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:10.050511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b4cee-8f9c-460d-b01c-8276abb5cde5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:10.051090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b4cee-8f9c-460d-b01c-8276abb5cde5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:10.051549 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b4cee-8f9c-460d-b01c-8276abb5cde5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:10.085902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d27b4cee-8f9c-460d-b01c-8276abb5cde5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:10.087107 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d27b4cee-8f9c-460d-b01c-8276abb5cde5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:10.181334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f94944a-e077-4a32-90e8-8f962ad57faf None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:10.196703 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:10.226996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 122/253] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:10.234255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eae4a4-cda5-4cd7-a933-ad02d704934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:10.234889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae4a4-cda5-4cd7-a933-ad02d704934a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:10.235170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae4a4-cda5-4cd7-a933-ad02d704934a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:10.235506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eae4a4-cda5-4cd7-a933-ad02d704934a None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:10.236181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eae4a4-cda5-4cd7-a933-ad02d704934a None admin] RBAC: Adding query filter params (name=member) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:10.236537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eae4a4-cda5-4cd7-a933-ad02d704934a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:10.238002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eae4a4-cda5-4cd7-a933-ad02d704934a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:10.252650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 132/254] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:10.258339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b4cee-8f9c-460d-b01c-8276abb5cde5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:10.392882 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b4cee-8f9c-460d-b01c-8276abb5cde5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:10.436460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f25d0-49fc-47b0-a940-aa054b7a1f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:10.437523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f25d0-49fc-47b0-a940-aa054b7a1f50 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:10.438000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f25d0-49fc-47b0-a940-aa054b7a1f50 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:10.438472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051f25d0-49fc-47b0-a940-aa054b7a1f50 None admin] PATH_INFO: `/v3/users/4420a089cd6740dc809c1e9ce9583b38` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:10.439827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f25d0-49fc-47b0-a940-aa054b7a1f50 None admin] RBAC: Authorizing `identity:get_user(user_id=4420a089cd6740dc809c1e9ce9583b38)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:10.441510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051f25d0-49fc-47b0-a940-aa054b7a1f50 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:10.444045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 123/255] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:10 2022] GET /identity/v3/users/4420a089cd6740dc809c1e9ce9583b38 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:10.449724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eae4a4-cda5-4cd7-a933-ad02d704934a None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:10.515278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:10.558915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f25d0-49fc-47b0-a940-aa054b7a1f50 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:10.603429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eae4a4-cda5-4cd7-a933-ad02d704934a None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:10.660486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:10.673098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc76966-de64-432b-ada9-47e40c0b07b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:10.673962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc76966-de64-432b-ada9-47e40c0b07b3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:10.674373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc76966-de64-432b-ada9-47e40c0b07b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:10.674775 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc76966-de64-432b-ada9-47e40c0b07b3 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:10.676201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc76966-de64-432b-ada9-47e40c0b07b3 None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:10.678528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc76966-de64-432b-ada9-47e40c0b07b3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:10.684028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 133/256] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:10 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:10.689838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc76966-de64-432b-ada9-47e40c0b07b3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:10.765171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051f25d0-49fc-47b0-a940-aa054b7a1f50 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:10.794249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc76966-de64-432b-ada9-47e40c0b07b3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:10.825484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:10.834812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d754d2f-5ccc-4f20-9053-edb9790e304f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:10.835452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d754d2f-5ccc-4f20-9053-edb9790e304f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:10.835730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d754d2f-5ccc-4f20-9053-edb9790e304f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:10.836002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d754d2f-5ccc-4f20-9053-edb9790e304f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:10.851675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850164f-681c-447b-9069-f5b0257b8e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:10.852333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850164f-681c-447b-9069-f5b0257b8e0d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:10.852615 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850164f-681c-447b-9069-f5b0257b8e0d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:10.852915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a850164f-681c-447b-9069-f5b0257b8e0d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:10.862204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d754d2f-5ccc-4f20-9053-edb9790e304f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-8d754d2f-5ccc-4f20-9053-edb9790e304f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:10.868711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:10.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:10.870951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 134/257] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:29:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:10.879600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d754d2f-5ccc-4f20-9053-edb9790e304f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:10.881354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a850164f-681c-447b-9069-f5b0257b8e0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a850164f-681c-447b-9069-f5b0257b8e0d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:10.887361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:10.888204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:10.889044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:10.902553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 124/258] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:29:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:10.903514 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850164f-681c-447b-9069-f5b0257b8e0d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:11.056711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d754d2f-5ccc-4f20-9053-edb9790e304f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:11.068396 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a850164f-681c-447b-9069-f5b0257b8e0d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:11.093457 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3b5af-a0c8-4ba1-a6f3-35dcbfe4acfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:11.094118 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b5af-a0c8-4ba1-a6f3-35dcbfe4acfd None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:11.094393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b5af-a0c8-4ba1-a6f3-35dcbfe4acfd None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:11.094669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac3b5af-a0c8-4ba1-a6f3-35dcbfe4acfd None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22/users/4420a089cd6740dc809c1e9ce9583b38/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:11.096666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3b5af-a0c8-4ba1-a6f3-35dcbfe4acfd None admin] RBAC: Authorizing `identity:create_grant(project_id=ef03f831e4b94427a2be5482f556af22, user_id=4420a089cd6740dc809c1e9ce9583b38, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:11.098160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac3b5af-a0c8-4ba1-a6f3-35dcbfe4acfd None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:11.102064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:11.118442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 135/259] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:29:10 2022] PUT /identity/v3/projects/ef03f831e4b94427a2be5482f556af22/users/4420a089cd6740dc809c1e9ce9583b38/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:11.119322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3b5af-a0c8-4ba1-a6f3-35dcbfe4acfd None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:11.172990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:11.186256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac8fb2-24eb-44a0-8041-9334c526aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:11.186886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac8fb2-24eb-44a0-8041-9334c526aa07 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:11.187294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac8fb2-24eb-44a0-8041-9334c526aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:11.187438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ac8fb2-24eb-44a0-8041-9334c526aa07 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:11.188101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac8fb2-24eb-44a0-8041-9334c526aa07 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:11.188401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac8fb2-24eb-44a0-8041-9334c526aa07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:11.189822 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ac8fb2-24eb-44a0-8041-9334c526aa07 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:11.214753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 125/260] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:11.221828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac8fb2-24eb-44a0-8041-9334c526aa07 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:11.307015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac3b5af-a0c8-4ba1-a6f3-35dcbfe4acfd None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:11.381415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bd507-8c6c-4373-bd00-0d7b6f338ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:11.382555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bd507-8c6c-4373-bd00-0d7b6f338ca0 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:11.382977 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bd507-8c6c-4373-bd00-0d7b6f338ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:11.383389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90bd507-8c6c-4373-bd00-0d7b6f338ca0 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:11.384280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bd507-8c6c-4373-bd00-0d7b6f338ca0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:11.384793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bd507-8c6c-4373-bd00-0d7b6f338ca0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:11.386878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90bd507-8c6c-4373-bd00-0d7b6f338ca0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:11.404966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 136/261] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:11.411023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bd507-8c6c-4373-bd00-0d7b6f338ca0 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:11.486326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ac8fb2-24eb-44a0-8041-9334c526aa07 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:11.561646 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a05ee-824f-4a48-9fa2-8d577480a101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:11.562759 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:11.563863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a05ee-824f-4a48-9fa2-8d577480a101 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:11.564308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a05ee-824f-4a48-9fa2-8d577480a101 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:11.565793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a05ee-824f-4a48-9fa2-8d577480a101 None admin] PATH_INFO: `/v3/users/9aeb4f850cff4f92ab5fe65a8665a622` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:11.567345 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a05ee-824f-4a48-9fa2-8d577480a101 None admin] RBAC: Authorizing `identity:get_user(user_id=9aeb4f850cff4f92ab5fe65a8665a622)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:11.570360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a05ee-824f-4a48-9fa2-8d577480a101 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:11.573279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 126/262] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:11 2022] GET /identity/v3/users/9aeb4f850cff4f92ab5fe65a8665a622 => generated 283 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:11.581435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a05ee-824f-4a48-9fa2-8d577480a101 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:11.591220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90bd507-8c6c-4373-bd00-0d7b6f338ca0 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:11.663321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bdd9b-7ffb-4a3c-8fca-3aaf49eb1beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:11.664269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bdd9b-7ffb-4a3c-8fca-3aaf49eb1beb None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:11.664690 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bdd9b-7ffb-4a3c-8fca-3aaf49eb1beb None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:11.665195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791bdd9b-7ffb-4a3c-8fca-3aaf49eb1beb None admin] PATH_INFO: `/v3/users/9aeb4f850cff4f92ab5fe65a8665a622` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:11.666997 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bdd9b-7ffb-4a3c-8fca-3aaf49eb1beb None admin] RBAC: Authorizing `identity:get_user(user_id=9aeb4f850cff4f92ab5fe65a8665a622)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:11.669736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791bdd9b-7ffb-4a3c-8fca-3aaf49eb1beb None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:11.673676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:11.675279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 137/263] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:11 2022] GET /identity/v3/users/9aeb4f850cff4f92ab5fe65a8665a622 => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:11.681505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bdd9b-7ffb-4a3c-8fca-3aaf49eb1beb None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:11.808593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a05ee-824f-4a48-9fa2-8d577480a101 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:11.819551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791bdd9b-7ffb-4a3c-8fca-3aaf49eb1beb None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:11.893484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87456738-9189-4ac9-9fb5-d13e73a150d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:11.894347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87456738-9189-4ac9-9fb5-d13e73a150d6 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:11.894803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87456738-9189-4ac9-9fb5-d13e73a150d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:11.895223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87456738-9189-4ac9-9fb5-d13e73a150d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:11.902876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf5565-fe48-4882-b1e4-fcff4a7eb543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:11.903497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf5565-fe48-4882-b1e4-fcff4a7eb543 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:11.903776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf5565-fe48-4882-b1e4-fcff4a7eb543 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:11.904046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf5565-fe48-4882-b1e4-fcff4a7eb543 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:11.945152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:12.041002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87456738-9189-4ac9-9fb5-d13e73a150d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:12.042696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87456738-9189-4ac9-9fb5-d13e73a150d6 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:12.073849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf5565-fe48-4882-b1e4-fcff4a7eb543 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:12.075233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bf5565-fe48-4882-b1e4-fcff4a7eb543 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:12.179044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:12.264480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 138/264] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 583 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:12.270352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87456738-9189-4ac9-9fb5-d13e73a150d6 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:12.282050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 127/265] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 22:29:12.290581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf5565-fe48-4882-b1e4-fcff4a7eb543 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:12.353291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:12.401127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87456738-9189-4ac9-9fb5-d13e73a150d6 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:12.461551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c207a-611a-4eac-9eaa-d39a71ec3d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:12.462214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c207a-611a-4eac-9eaa-d39a71ec3d1b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:12.462492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c207a-611a-4eac-9eaa-d39a71ec3d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:12.462766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c207a-611a-4eac-9eaa-d39a71ec3d1b None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:12.464728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c207a-611a-4eac-9eaa-d39a71ec3d1b None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:12.466253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c207a-611a-4eac-9eaa-d39a71ec3d1b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:12.468197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 139/266] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:12 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:12.470822 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf5565-fe48-4882-b1e4-fcff4a7eb543 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:12.475366 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c207a-611a-4eac-9eaa-d39a71ec3d1b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:12.515706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fd78d0-4230-4ab8-acea-8118afb36d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:12.516434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd78d0-4230-4ab8-acea-8118afb36d33 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:12.516714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd78d0-4230-4ab8-acea-8118afb36d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:12.517003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fd78d0-4230-4ab8-acea-8118afb36d33 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:12.518233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd78d0-4230-4ab8-acea-8118afb36d33 None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:12.519682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fd78d0-4230-4ab8-acea-8118afb36d33 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:12.526399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 128/267] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:12 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:12.527737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c116e89-f71d-4a09-b72c-9da22a5726d7 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:12.528021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c116e89-f71d-4a09-b72c-9da22a5726d7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:12.528289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c116e89-f71d-4a09-b72c-9da22a5726d7 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:12.529743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 129/268] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:12.530643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c116e89-f71d-4a09-b72c-9da22a5726d7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:12.546445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:12.642222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c116e89-f71d-4a09-b72c-9da22a5726d7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:12.681962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b9be9-4c79-4222-8dac-e6af0e170a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:12.682563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b9be9-4c79-4222-8dac-e6af0e170a0c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:12.682832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b9be9-4c79-4222-8dac-e6af0e170a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:12.683095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b9be9-4c79-4222-8dac-e6af0e170a0c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:12.683813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b9be9-4c79-4222-8dac-e6af0e170a0c None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=ef03f831e4b94427a2be5482f556af22, user.id=9aeb4f850cff4f92ab5fe65a8665a622) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:12.684112 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b9be9-4c79-4222-8dac-e6af0e170a0c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:12.685568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b9be9-4c79-4222-8dac-e6af0e170a0c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:12.696881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 130/269] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:12 2022] GET /identity/v3/role_assignments?user.id=9aeb4f850cff4f92ab5fe65a8665a622&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:12.698243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3251775-1b4c-48d8-8dc0-0331b6c1d0db None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:12.698524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3251775-1b4c-48d8-8dc0-0331b6c1d0db None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:12.698792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3251775-1b4c-48d8-8dc0-0331b6c1d0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:12.733989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e3251775-1b4c-48d8-8dc0-0331b6c1d0db None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:12.735216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e3251775-1b4c-48d8-8dc0-0331b6c1d0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:12.746923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c207a-611a-4eac-9eaa-d39a71ec3d1b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:12.805286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:12.828479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e6d9ff-e8b9-451a-a0b8-ac33cc79a3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:12.829134 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6d9ff-e8b9-451a-a0b8-ac33cc79a3fd None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:12.829467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6d9ff-e8b9-451a-a0b8-ac33cc79a3fd None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:12.829738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e6d9ff-e8b9-451a-a0b8-ac33cc79a3fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:12.830526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6d9ff-e8b9-451a-a0b8-ac33cc79a3fd None admin] RBAC: Adding query filter params (role.id=952084593c184a78a952e1797a0b6c6b, scope.project.id=ef03f831e4b94427a2be5482f556af22, user.id=9aeb4f850cff4f92ab5fe65a8665a622) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:12.830834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6d9ff-e8b9-451a-a0b8-ac33cc79a3fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:12.832329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e6d9ff-e8b9-451a-a0b8-ac33cc79a3fd None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:12.839608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 140/270] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:12 2022] GET /identity/v3/role_assignments?user.id=9aeb4f850cff4f92ab5fe65a8665a622&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=952084593c184a78a952e1797a0b6c6b => generated 608 bytes in 365 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:12.840967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c93530-d4b2-443e-ac7d-449b49f7b665 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:12.841259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c93530-d4b2-443e-ac7d-449b49f7b665 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:12.841538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c93530-d4b2-443e-ac7d-449b49f7b665 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:12.843097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 141/271] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:12.854255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1cc4a-99b3-4677-a6f5-28f823c5750a None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:12.854555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1cc4a-99b3-4677-a6f5-28f823c5750a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:12.854790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f1cc4a-99b3-4677-a6f5-28f823c5750a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:12.888215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e7f1cc4a-99b3-4677-a6f5-28f823c5750a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:12.889501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e7f1cc4a-99b3-4677-a6f5-28f823c5750a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:12.894498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 131/272] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:12.993297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 142/273] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:13.206757 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3251775-1b4c-48d8-8dc0-0331b6c1d0db None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:13.217110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:13.327081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3251775-1b4c-48d8-8dc0-0331b6c1d0db None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:13.340179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964c61d-54a8-4f6a-b872-0141317929f1 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:13.340714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964c61d-54a8-4f6a-b872-0141317929f1 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:13.340714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f964c61d-54a8-4f6a-b872-0141317929f1 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:13.342343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 143/274] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:13.347033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e720e-1043-494f-9fea-fa7d470463ce None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:13.347317 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e720e-1043-494f-9fea-fa7d470463ce None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:13.347586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e720e-1043-494f-9fea-fa7d470463ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:13.371175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-271e720e-1043-494f-9fea-fa7d470463ce None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:13.372534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-271e720e-1043-494f-9fea-fa7d470463ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:13.387302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84bb1c-30f0-42d6-a884-a23e883017b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:13.388108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84bb1c-30f0-42d6-a884-a23e883017b9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:13.388384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84bb1c-30f0-42d6-a884-a23e883017b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:13.388653 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f84bb1c-30f0-42d6-a884-a23e883017b9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:13.390188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84bb1c-30f0-42d6-a884-a23e883017b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:13.391729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f84bb1c-30f0-42d6-a884-a23e883017b9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:13.393370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 132/275] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:13.396837 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84bb1c-30f0-42d6-a884-a23e883017b9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:13.456594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 144/276] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:13.457730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e720e-1043-494f-9fea-fa7d470463ce None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:13.482769 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:13.521848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f84bb1c-30f0-42d6-a884-a23e883017b9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:13.531312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:13.573647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e112a0-1b4f-4279-80be-b3b4054ebb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:13.573647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e112a0-1b4f-4279-80be-b3b4054ebb40 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:13.575050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e112a0-1b4f-4279-80be-b3b4054ebb40 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:13.575448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e112a0-1b4f-4279-80be-b3b4054ebb40 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:13.592744 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e112a0-1b4f-4279-80be-b3b4054ebb40 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-76e112a0-1b4f-4279-80be-b3b4054ebb40 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:13.599630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:13.600737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:13.601752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:13.603456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 133/277] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:13.608174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e720e-1043-494f-9fea-fa7d470463ce None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:13.609414 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e112a0-1b4f-4279-80be-b3b4054ebb40 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:13.654569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafa51a-fd50-433f-a008-71671f3b5161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:13.655875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafa51a-fd50-433f-a008-71671f3b5161 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:13.656308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafa51a-fd50-433f-a008-71671f3b5161 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:13.656780 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafa51a-fd50-433f-a008-71671f3b5161 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:13.657995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafa51a-fd50-433f-a008-71671f3b5161 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:13.660186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bafa51a-fd50-433f-a008-71671f3b5161 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:13.697570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 145/278] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:29:13 2022] POST /identity/v3/users => generated 293 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:29:13.726951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e112a0-1b4f-4279-80be-b3b4054ebb40 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:13.730368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafa51a-fd50-433f-a008-71671f3b5161 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:13.795236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2417741-ecf6-4b2a-be96-c961a94c8d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:13.795865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2417741-ecf6-4b2a-be96-c961a94c8d46 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:13.796145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2417741-ecf6-4b2a-be96-c961a94c8d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:13.796417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2417741-ecf6-4b2a-be96-c961a94c8d46 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:13.797129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2417741-ecf6-4b2a-be96-c961a94c8d46 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:13.797430 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2417741-ecf6-4b2a-be96-c961a94c8d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:13.799006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2417741-ecf6-4b2a-be96-c961a94c8d46 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:13.801010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:13.811711 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 134/279] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:13.817752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2417741-ecf6-4b2a-be96-c961a94c8d46 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:13.850206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafa51a-fd50-433f-a008-71671f3b5161 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:13.853072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:13.888274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d662a-a856-4e72-b494-7973f96ce428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:13.888949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d662a-a856-4e72-b494-7973f96ce428 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:13.889224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d662a-a856-4e72-b494-7973f96ce428 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:13.889492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406d662a-a856-4e72-b494-7973f96ce428 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:13.901784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406d662a-a856-4e72-b494-7973f96ce428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-406d662a-a856-4e72-b494-7973f96ce428 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:13.907448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:13.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:13.909287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:13.914595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 146/280] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:13.915654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d662a-a856-4e72-b494-7973f96ce428 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:13.951249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2417741-ecf6-4b2a-be96-c961a94c8d46 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:14.003671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e4a400-73a5-442f-80b1-eb3c784ce600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:14.005989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4a400-73a5-442f-80b1-eb3c784ce600 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:14.006434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4a400-73a5-442f-80b1-eb3c784ce600 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:14.007461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e4a400-73a5-442f-80b1-eb3c784ce600 None admin] PATH_INFO: `/v3/users/a3397e6a4cc5458e9262e07d4fffead8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:14.026556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4a400-73a5-442f-80b1-eb3c784ce600 None admin] RBAC: Authorizing `identity:get_user(user_id=a3397e6a4cc5458e9262e07d4fffead8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:14.028262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e4a400-73a5-442f-80b1-eb3c784ce600 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:14.034845 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406d662a-a856-4e72-b494-7973f96ce428 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:14.035031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 135/281] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:13 2022] GET /identity/v3/users/a3397e6a4cc5458e9262e07d4fffead8 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:14.037053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac188915-6551-4199-bc4c-f13a4c31e71b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:14.037339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac188915-6551-4199-bc4c-f13a4c31e71b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:14.037615 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac188915-6551-4199-bc4c-f13a4c31e71b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:14.039108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 136/282] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:14.039632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac188915-6551-4199-bc4c-f13a4c31e71b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:14.072471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c4a77a-b699-4983-9725-63496e8b581b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:14.073434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4a77a-b699-4983-9725-63496e8b581b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:14.073847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4a77a-b699-4983-9725-63496e8b581b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:14.074251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c4a77a-b699-4983-9725-63496e8b581b None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:14.075213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4a77a-b699-4983-9725-63496e8b581b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:14.075652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4a77a-b699-4983-9725-63496e8b581b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:14.077414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c4a77a-b699-4983-9725-63496e8b581b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:14.088185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 147/283] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:13 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:14.089765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985960f-9c46-4371-a068-32a1468f62f2 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:14.090182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985960f-9c46-4371-a068-32a1468f62f2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:14.090611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1985960f-9c46-4371-a068-32a1468f62f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:14.104291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:14.115246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-1985960f-9c46-4371-a068-32a1468f62f2 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:14.116439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1985960f-9c46-4371-a068-32a1468f62f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:14.139429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:14.166829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac188915-6551-4199-bc4c-f13a4c31e71b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:14.201584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 148/284] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:14.202495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985960f-9c46-4371-a068-32a1468f62f2 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:14.225258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698796d5-c4b1-426e-aa13-26da0ff00c8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:14.227141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698796d5-c4b1-426e-aa13-26da0ff00c8e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:14.227419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698796d5-c4b1-426e-aa13-26da0ff00c8e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:14.227685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698796d5-c4b1-426e-aa13-26da0ff00c8e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:14.228409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698796d5-c4b1-426e-aa13-26da0ff00c8e None admin] RBAC: Adding query filter params (role.id=aac9ca64da09463ba9b3966c7188d7ee, user.id=a3397e6a4cc5458e9262e07d4fffead8, scope.system=all) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:14.228706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698796d5-c4b1-426e-aa13-26da0ff00c8e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:14.230161 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698796d5-c4b1-426e-aa13-26da0ff00c8e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:14.242300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 137/285] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Sep 6 22:29:14 2022] GET /identity/v3/role_assignments?user.id=a3397e6a4cc5458e9262e07d4fffead8&scope.system=all&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:14.359234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1985960f-9c46-4371-a068-32a1468f62f2 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:14.397430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738248b-5311-46a3-af63-68bd8c7565b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:14.398226 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738248b-5311-46a3-af63-68bd8c7565b3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:14.398603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738248b-5311-46a3-af63-68bd8c7565b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:14.398993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6738248b-5311-46a3-af63-68bd8c7565b3 None admin] PATH_INFO: `/v3/users/c9a69256fd19439a8c5ce59f32871fb6` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:14.400274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738248b-5311-46a3-af63-68bd8c7565b3 None admin] RBAC: Authorizing `identity:get_user(user_id=c9a69256fd19439a8c5ce59f32871fb6)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:14.401864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6738248b-5311-46a3-af63-68bd8c7565b3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:14.403908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 149/286] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:14 2022] GET /identity/v3/users/c9a69256fd19439a8c5ce59f32871fb6 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:14.409153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698796d5-c4b1-426e-aa13-26da0ff00c8e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:14.523645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738248b-5311-46a3-af63-68bd8c7565b3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:14.529103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698796d5-c4b1-426e-aa13-26da0ff00c8e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:14.551179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:14.561875 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:14.582025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca0ff5-89fd-4003-a2cf-d9e227848eaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:14.583471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca0ff5-89fd-4003-a2cf-d9e227848eaf None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:14.583948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca0ff5-89fd-4003-a2cf-d9e227848eaf None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:14.584413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ca0ff5-89fd-4003-a2cf-d9e227848eaf None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:14.585776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca0ff5-89fd-4003-a2cf-d9e227848eaf None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:14.587462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8ca0ff5-89fd-4003-a2cf-d9e227848eaf None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:14.590493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 138/287] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:14 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:14.601866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca0ff5-89fd-4003-a2cf-d9e227848eaf None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:14.634607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6738248b-5311-46a3-af63-68bd8c7565b3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:14.708924 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1ef43-7b32-43b3-87ee-fb2e234e4097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:14.709570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1ef43-7b32-43b3-87ee-fb2e234e4097 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:14.709898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1ef43-7b32-43b3-87ee-fb2e234e4097 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:14.710168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe1ef43-7b32-43b3-87ee-fb2e234e4097 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:14.730210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe1ef43-7b32-43b3-87ee-fb2e234e4097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-ffe1ef43-7b32-43b3-87ee-fb2e234e4097 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:14.737883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:14.738799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:14.740202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 150/288] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:14.746658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1ef43-7b32-43b3-87ee-fb2e234e4097 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:14.830698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ca0ff5-89fd-4003-a2cf-d9e227848eaf None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:14.871585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe1ef43-7b32-43b3-87ee-fb2e234e4097 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:14.894246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:14.914097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed81b4e-bb91-49b8-a0c1-0382015dbad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:14.914835 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed81b4e-bb91-49b8-a0c1-0382015dbad7 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:14.915141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed81b4e-bb91-49b8-a0c1-0382015dbad7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:14.915777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed81b4e-bb91-49b8-a0c1-0382015dbad7 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22/users/c9a69256fd19439a8c5ce59f32871fb6/roles/4f87bf3cce2d4008a95c1da7369d74bf` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:14.918109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed81b4e-bb91-49b8-a0c1-0382015dbad7 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef03f831e4b94427a2be5482f556af22, user_id=c9a69256fd19439a8c5ce59f32871fb6, role_id=4f87bf3cce2d4008a95c1da7369d74bf)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:14.920490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed81b4e-bb91-49b8-a0c1-0382015dbad7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:14.921627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87832bdc-2c38-416a-881e-6c1e929ab723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:14.922478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832bdc-2c38-416a-881e-6c1e929ab723 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:14.923132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832bdc-2c38-416a-881e-6c1e929ab723 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:14.923600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87832bdc-2c38-416a-881e-6c1e929ab723 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:14.925141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87832bdc-2c38-416a-881e-6c1e929ab723 None admin] RBAC: Adding query filter params (name=member) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:14.929009 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87832bdc-2c38-416a-881e-6c1e929ab723 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:14.930677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87832bdc-2c38-416a-881e-6c1e929ab723 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:14.937717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 139/289] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:29:14 2022] PUT /identity/v3/projects/ef03f831e4b94427a2be5482f556af22/users/c9a69256fd19439a8c5ce59f32871fb6/roles/4f87bf3cce2d4008a95c1da7369d74bf => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:14.955416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 151/290] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:14.961897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed81b4e-bb91-49b8-a0c1-0382015dbad7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:15.046374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:15.128699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed81b4e-bb91-49b8-a0c1-0382015dbad7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:15.176487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434729-ed03-4593-a9f1-42ed4777cb87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:15.177716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434729-ed03-4593-a9f1-42ed4777cb87 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:15.177982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434729-ed03-4593-a9f1-42ed4777cb87 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:15.178248 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33434729-ed03-4593-a9f1-42ed4777cb87 None admin] PATH_INFO: `/v3/users/4420a089cd6740dc809c1e9ce9583b38` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:15.179416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33434729-ed03-4593-a9f1-42ed4777cb87 None admin] RBAC: Authorizing `identity:get_user(user_id=4420a089cd6740dc809c1e9ce9583b38)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:15.180896 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33434729-ed03-4593-a9f1-42ed4777cb87 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:15.182779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 140/291] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:14 2022] GET /identity/v3/users/4420a089cd6740dc809c1e9ce9583b38 => generated 297 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:15.189872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87832bdc-2c38-416a-881e-6c1e929ab723 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:15.313115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87832bdc-2c38-416a-881e-6c1e929ab723 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:15.356420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df939b04-6ce5-45d7-ad0f-0291248928ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:15.357407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df939b04-6ce5-45d7-ad0f-0291248928ba None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:15.357944 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df939b04-6ce5-45d7-ad0f-0291248928ba None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:15.358434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df939b04-6ce5-45d7-ad0f-0291248928ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:15.384082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:15.454859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df939b04-6ce5-45d7-ad0f-0291248928ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:15.456230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df939b04-6ce5-45d7-ad0f-0291248928ba None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:15.587799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 152/292] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:15.593682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434729-ed03-4593-a9f1-42ed4777cb87 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:15.744114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33434729-ed03-4593-a9f1-42ed4777cb87 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:15.762295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:15.789941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e943baf5-8b93-40fe-b000-303b647f2d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:15.791977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943baf5-8b93-40fe-b000-303b647f2d16 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:15.792247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943baf5-8b93-40fe-b000-303b647f2d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:15.792516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e943baf5-8b93-40fe-b000-303b647f2d16 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:15.797440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943baf5-8b93-40fe-b000-303b647f2d16 None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:15.798881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e943baf5-8b93-40fe-b000-303b647f2d16 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:15.800510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 141/293] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:15 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:15.808417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df939b04-6ce5-45d7-ad0f-0291248928ba None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:15.863901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:15.921105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df939b04-6ce5-45d7-ad0f-0291248928ba None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:15.957283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ad2f4-b46d-4cbc-a1d6-24946b72dea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:15.958413 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ad2f4-b46d-4cbc-a1d6-24946b72dea3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:15.958913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ad2f4-b46d-4cbc-a1d6-24946b72dea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:15.959406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ad2f4-b46d-4cbc-a1d6-24946b72dea3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:15.960320 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ad2f4-b46d-4cbc-a1d6-24946b72dea3 None admin] RBAC: Adding query filter params (role.id=aac9ca64da09463ba9b3966c7188d7ee, scope.project.id=ef03f831e4b94427a2be5482f556af22, user.id=4420a089cd6740dc809c1e9ce9583b38) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:15.960864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ad2f4-b46d-4cbc-a1d6-24946b72dea3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:15.962474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ad2f4-b46d-4cbc-a1d6-24946b72dea3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:15.968695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 153/294] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:15 2022] GET /identity/v3/role_assignments?user.id=4420a089cd6740dc809c1e9ce9583b38&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:16.447931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ce5f9-73a8-481e-bc35-3f5de16c7aeb None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:16.448336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ce5f9-73a8-481e-bc35-3f5de16c7aeb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:16.448516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787ce5f9-73a8-481e-bc35-3f5de16c7aeb None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:16.449939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 142/295] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:16.455262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb15f2-551a-497c-9df7-6c08c1a8407c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:16.455716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb15f2-551a-497c-9df7-6c08c1a8407c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:16.455793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb15f2-551a-497c-9df7-6c08c1a8407c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:16.475494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-08bb15f2-551a-497c-9df7-6c08c1a8407c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:16.476683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-08bb15f2-551a-497c-9df7-6c08c1a8407c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:16.564937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 154/296] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:16.767890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ca84-4f5e-4edb-98c1-a38de84bfb5f None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:16.768355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ca84-4f5e-4edb-98c1-a38de84bfb5f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:16.768355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4ca84-4f5e-4edb-98c1-a38de84bfb5f None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:16.769956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 143/297] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:16.782173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436af1a9-714c-4e59-8aa2-9e2cbae7429b None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:16.782734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436af1a9-714c-4e59-8aa2-9e2cbae7429b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:16.783214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436af1a9-714c-4e59-8aa2-9e2cbae7429b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:16.805681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-436af1a9-714c-4e59-8aa2-9e2cbae7429b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:16.807212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-436af1a9-714c-4e59-8aa2-9e2cbae7429b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:16.834121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3da54-4205-4f3a-bb96-6cd0ef455dd7 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:16.834405 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3da54-4205-4f3a-bb96-6cd0ef455dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:16.834659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a3da54-4205-4f3a-bb96-6cd0ef455dd7 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:16.836105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 144/298] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:29:16.841301 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16959129-cb12-492c-aee7-2836b328529f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:16.841580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16959129-cb12-492c-aee7-2836b328529f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:16.841853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16959129-cb12-492c-aee7-2836b328529f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:16.846446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:16.863335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-16959129-cb12-492c-aee7-2836b328529f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:16.864884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-16959129-cb12-492c-aee7-2836b328529f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:16.884731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 155/299] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:16.885523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436af1a9-714c-4e59-8aa2-9e2cbae7429b None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:16.957719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 145/300] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:16.995220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436af1a9-714c-4e59-8aa2-9e2cbae7429b None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:17.042744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40217d-f290-4051-835a-53d40b97475a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:17.043801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40217d-f290-4051-835a-53d40b97475a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:17.044491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40217d-f290-4051-835a-53d40b97475a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:17.045189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40217d-f290-4051-835a-53d40b97475a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:17.060136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40217d-f290-4051-835a-53d40b97475a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-7d40217d-f290-4051-835a-53d40b97475a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:17.066454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:17.068369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:17.069807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:17.077064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 156/301] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:17.079964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16959129-cb12-492c-aee7-2836b328529f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:17.159356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40217d-f290-4051-835a-53d40b97475a None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:17.196556 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:17.199562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16959129-cb12-492c-aee7-2836b328529f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:17.215746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:17.239346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd675e-e914-4a56-9b01-a0e849313c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:17.239945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd675e-e914-4a56-9b01-a0e849313c8c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:17.240229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd675e-e914-4a56-9b01-a0e849313c8c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:17.240498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cd675e-e914-4a56-9b01-a0e849313c8c None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:17.241275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd675e-e914-4a56-9b01-a0e849313c8c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:17.241576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd675e-e914-4a56-9b01-a0e849313c8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:17.242938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cd675e-e914-4a56-9b01-a0e849313c8c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:17.251613 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 146/302] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:17.252218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd675e-e914-4a56-9b01-a0e849313c8c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:17.300578 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40217d-f290-4051-835a-53d40b97475a None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:17.339018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf12999-0c99-4ed5-aa5d-bba294067d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:17.339977 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf12999-0c99-4ed5-aa5d-bba294067d43 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:17.340389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf12999-0c99-4ed5-aa5d-bba294067d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:17.340815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf12999-0c99-4ed5-aa5d-bba294067d43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:17.352111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf12999-0c99-4ed5-aa5d-bba294067d43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-cdf12999-0c99-4ed5-aa5d-bba294067d43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:17.357309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:17.358328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:17.359282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:17.360005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cd675e-e914-4a56-9b01-a0e849313c8c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:17.361397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 157/303] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:17.362498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf12999-0c99-4ed5-aa5d-bba294067d43 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:17.408389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32e7b7-28e4-4e37-8614-8212945b06a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:17.409395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32e7b7-28e4-4e37-8614-8212945b06a6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:17.409822 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32e7b7-28e4-4e37-8614-8212945b06a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:17.410233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb32e7b7-28e4-4e37-8614-8212945b06a6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:17.411633 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32e7b7-28e4-4e37-8614-8212945b06a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:17.413136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb32e7b7-28e4-4e37-8614-8212945b06a6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:17.416802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 147/304] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:17.416802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32e7b7-28e4-4e37-8614-8212945b06a6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:17.460913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:17.466449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:17.479339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf12999-0c99-4ed5-aa5d-bba294067d43 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:17.514667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440825f-fe9d-4148-aee7-060e9ff96f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:17.515284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440825f-fe9d-4148-aee7-060e9ff96f0a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:17.515557 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440825f-fe9d-4148-aee7-060e9ff96f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:17.515825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0440825f-fe9d-4148-aee7-060e9ff96f0a None admin] PATH_INFO: `/v3/users/e832d8c6868945b4916a1f28962f63c4` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:17.519484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb32e7b7-28e4-4e37-8614-8212945b06a6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:17.529820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440825f-fe9d-4148-aee7-060e9ff96f0a None admin] RBAC: Authorizing `identity:get_user(user_id=e832d8c6868945b4916a1f28962f63c4)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:17.531297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0440825f-fe9d-4148-aee7-060e9ff96f0a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:17.533244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 158/305] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:17 2022] GET /identity/v3/users/e832d8c6868945b4916a1f28962f63c4 => generated 293 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:17.533944 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440825f-fe9d-4148-aee7-060e9ff96f0a None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:17.567956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7931c-dbe8-4b19-b2f3-699a32be2b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:17.569064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7931c-dbe8-4b19-b2f3-699a32be2b07 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:17.569546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7931c-dbe8-4b19-b2f3-699a32be2b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:17.569996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b7931c-dbe8-4b19-b2f3-699a32be2b07 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:17.570856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7931c-dbe8-4b19-b2f3-699a32be2b07 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:17.571349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7931c-dbe8-4b19-b2f3-699a32be2b07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:17.572870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b7931c-dbe8-4b19-b2f3-699a32be2b07 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:17.581698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 148/306] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:17.586853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7931c-dbe8-4b19-b2f3-699a32be2b07 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:17.648264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0440825f-fe9d-4148-aee7-060e9ff96f0a None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:17.682350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65592cbe-b3d6-497d-90fc-324ba7953e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:17.682954 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65592cbe-b3d6-497d-90fc-324ba7953e34 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:17.683229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65592cbe-b3d6-497d-90fc-324ba7953e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:17.683494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65592cbe-b3d6-497d-90fc-324ba7953e34 None admin] PATH_INFO: `/v3/groups` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:17.684117 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65592cbe-b3d6-497d-90fc-324ba7953e34 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:17.685558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65592cbe-b3d6-497d-90fc-324ba7953e34 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:17.694028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b7931c-dbe8-4b19-b2f3-699a32be2b07 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:17.696818 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:17.702684 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 159/307] 10.222.0.39 () {60 vars in 1134 bytes} [Tue Sep 6 22:29:17 2022] POST /identity/v3/groups => generated 229 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:29:17.703479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65592cbe-b3d6-497d-90fc-324ba7953e34 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:17.725405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:17.735359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa861d6-88a0-468b-ae7a-7ef7a11c64cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:17.735985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa861d6-88a0-468b-ae7a-7ef7a11c64cf None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:17.736266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa861d6-88a0-468b-ae7a-7ef7a11c64cf None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:17.736603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa861d6-88a0-468b-ae7a-7ef7a11c64cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:17.737558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa861d6-88a0-468b-ae7a-7ef7a11c64cf None admin] RBAC: Adding query filter params (role.id=4f87bf3cce2d4008a95c1da7369d74bf, user.id=e832d8c6868945b4916a1f28962f63c4, scope.system=all) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:17.737558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa861d6-88a0-468b-ae7a-7ef7a11c64cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:17.739985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa861d6-88a0-468b-ae7a-7ef7a11c64cf None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:17.752854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 149/308] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Sep 6 22:29:17 2022] GET /identity/v3/role_assignments?user.id=e832d8c6868945b4916a1f28962f63c4&scope.system=all&role.id=4f87bf3cce2d4008a95c1da7369d74bf => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:17.752854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8acbb-be1c-4842-87c2-eae62c43a21e None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:17.752854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8acbb-be1c-4842-87c2-eae62c43a21e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:17.752854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a8acbb-be1c-4842-87c2-eae62c43a21e None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:17.752854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 150/309] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:17.759918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d7459-f144-4161-93a5-63c9b43a78f0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:17.760791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d7459-f144-4161-93a5-63c9b43a78f0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:17.761274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551d7459-f144-4161-93a5-63c9b43a78f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:17.788351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-551d7459-f144-4161-93a5-63c9b43a78f0 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:17.791388 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-551d7459-f144-4161-93a5-63c9b43a78f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:17.849051 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65592cbe-b3d6-497d-90fc-324ba7953e34 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:17.888919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969a16c-1ef2-4953-b08f-5bf7e5c6ecf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:17.889657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969a16c-1ef2-4953-b08f-5bf7e5c6ecf2 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:17.890001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969a16c-1ef2-4953-b08f-5bf7e5c6ecf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:17.890335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969a16c-1ef2-4953-b08f-5bf7e5c6ecf2 None admin] PATH_INFO: `/v3/users/a3397e6a4cc5458e9262e07d4fffead8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:17.891565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969a16c-1ef2-4953-b08f-5bf7e5c6ecf2 None admin] RBAC: Authorizing `identity:get_user(user_id=a3397e6a4cc5458e9262e07d4fffead8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:17.893243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 151/310] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:17.894552 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969a16c-1ef2-4953-b08f-5bf7e5c6ecf2 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:17.896470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 160/311] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:17 2022] GET /identity/v3/users/a3397e6a4cc5458e9262e07d4fffead8 => generated 293 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:17.902238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d7459-f144-4161-93a5-63c9b43a78f0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:18.014719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:18.018813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551d7459-f144-4161-93a5-63c9b43a78f0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:18.058638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891b043-ec4d-4c1a-9b61-039bdb2fe08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:18.059284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891b043-ec4d-4c1a-9b61-039bdb2fe08a None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:18.059554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891b043-ec4d-4c1a-9b61-039bdb2fe08a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:18.059819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a891b043-ec4d-4c1a-9b61-039bdb2fe08a None admin] PATH_INFO: `/v3/system/users/a3397e6a4cc5458e9262e07d4fffead8/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:18.061265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891b043-ec4d-4c1a-9b61-039bdb2fe08a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a3397e6a4cc5458e9262e07d4fffead8, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:18.062640 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a891b043-ec4d-4c1a-9b61-039bdb2fe08a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:18.074876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 152/312] 10.222.0.39 () {58 vars in 1334 bytes} [Tue Sep 6 22:29:17 2022] PUT /identity/v3/system/users/a3397e6a4cc5458e9262e07d4fffead8/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:29:18.184876 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969a16c-1ef2-4953-b08f-5bf7e5c6ecf2 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:18.274492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:18.300271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969a16c-1ef2-4953-b08f-5bf7e5c6ecf2 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:18.337963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6be4af-46bf-4521-b038-9b168fea0b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:18.338596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6be4af-46bf-4521-b038-9b168fea0b83 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:18.338872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6be4af-46bf-4521-b038-9b168fea0b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:18.339142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6be4af-46bf-4521-b038-9b168fea0b83 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:18.350911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6be4af-46bf-4521-b038-9b168fea0b83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-de6be4af-46bf-4521-b038-9b168fea0b83 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:18.356958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:18.357844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:18.358645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 161/313] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:18 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:18.366493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891b043-ec4d-4c1a-9b61-039bdb2fe08a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:18.471469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a891b043-ec4d-4c1a-9b61-039bdb2fe08a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:18.506902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f35ae9-6a1e-43c2-bfea-2e9fc6893d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:18.507773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f35ae9-6a1e-43c2-bfea-2e9fc6893d5e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:18.508043 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f35ae9-6a1e-43c2-bfea-2e9fc6893d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:18.508308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f35ae9-6a1e-43c2-bfea-2e9fc6893d5e None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:18.508990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f35ae9-6a1e-43c2-bfea-2e9fc6893d5e None admin] RBAC: Adding query filter params (name=reader) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:18.509290 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f35ae9-6a1e-43c2-bfea-2e9fc6893d5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:18.510641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f35ae9-6a1e-43c2-bfea-2e9fc6893d5e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:18.519190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 153/314] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:18 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:18.524895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6be4af-46bf-4521-b038-9b168fea0b83 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:18.633791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6be4af-46bf-4521-b038-9b168fea0b83 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:18.671173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c92c82-884c-4b97-bcb6-c19c195892f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:18.671804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c92c82-884c-4b97-bcb6-c19c195892f0 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:18.672081 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c92c82-884c-4b97-bcb6-c19c195892f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:18.672355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c92c82-884c-4b97-bcb6-c19c195892f0 None admin] PATH_INFO: `/v3/users/c9a69256fd19439a8c5ce59f32871fb6` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:18.673683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c92c82-884c-4b97-bcb6-c19c195892f0 None admin] RBAC: Authorizing `identity:get_user(user_id=c9a69256fd19439a8c5ce59f32871fb6)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:18.675133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c92c82-884c-4b97-bcb6-c19c195892f0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:18.677431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 162/315] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:18 2022] GET /identity/v3/users/c9a69256fd19439a8c5ce59f32871fb6 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:18.682835 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f35ae9-6a1e-43c2-bfea-2e9fc6893d5e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:18.710053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:18.803809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f35ae9-6a1e-43c2-bfea-2e9fc6893d5e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:18.842119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee76807-c12f-43d6-a573-9efded973d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:18.842737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee76807-c12f-43d6-a573-9efded973d55 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:18.843012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee76807-c12f-43d6-a573-9efded973d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:18.843280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee76807-c12f-43d6-a573-9efded973d55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:18.915904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee76807-c12f-43d6-a573-9efded973d55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:18.917416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee76807-c12f-43d6-a573-9efded973d55 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:18.943239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:19.028193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 154/316] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:19.033876 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c92c82-884c-4b97-bcb6-c19c195892f0 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:19.038074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:19.151360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c92c82-884c-4b97-bcb6-c19c195892f0 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:19.186999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afb20b1-6ab8-4ddc-abd5-d68f30a5dd1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:19.187628 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb20b1-6ab8-4ddc-abd5-d68f30a5dd1c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:19.187905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb20b1-6ab8-4ddc-abd5-d68f30a5dd1c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:19.188176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afb20b1-6ab8-4ddc-abd5-d68f30a5dd1c None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:19.189330 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb20b1-6ab8-4ddc-abd5-d68f30a5dd1c None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:19.190766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afb20b1-6ab8-4ddc-abd5-d68f30a5dd1c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:19.193544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 163/317] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:19 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:19.199410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee76807-c12f-43d6-a573-9efded973d55 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:19.304525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee76807-c12f-43d6-a573-9efded973d55 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:19.328845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:19.342082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb723a7a-a9e3-4d29-b1c8-67ec0465f8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:19.342685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723a7a-a9e3-4d29-b1c8-67ec0465f8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:19.342963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723a7a-a9e3-4d29-b1c8-67ec0465f8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:19.343233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb723a7a-a9e3-4d29-b1c8-67ec0465f8b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:19.343953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb723a7a-a9e3-4d29-b1c8-67ec0465f8b6 None admin] RBAC: Adding query filter params (role.id=4f87bf3cce2d4008a95c1da7369d74bf, scope.project.id=ef03f831e4b94427a2be5482f556af22, user.id=c9a69256fd19439a8c5ce59f32871fb6) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:19.344256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb723a7a-a9e3-4d29-b1c8-67ec0465f8b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:19.345733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb723a7a-a9e3-4d29-b1c8-67ec0465f8b6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:19.352044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 155/318] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:19 2022] GET /identity/v3/role_assignments?user.id=c9a69256fd19439a8c5ce59f32871fb6&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=4f87bf3cce2d4008a95c1da7369d74bf => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:20.054888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d6087-bfba-43b0-a56b-a8a2f90cedf5 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:20.055400 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d6087-bfba-43b0-a56b-a8a2f90cedf5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:20.055400 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d6087-bfba-43b0-a56b-a8a2f90cedf5 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:20.056740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 164/319] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:20.061947 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25805095-f499-458d-9e8e-8b2c63bf893a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:20.062323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25805095-f499-458d-9e8e-8b2c63bf893a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:20.062482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25805095-f499-458d-9e8e-8b2c63bf893a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:20.081412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-25805095-f499-458d-9e8e-8b2c63bf893a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:20.082572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-25805095-f499-458d-9e8e-8b2c63bf893a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:20.137892 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9c035-7f06-4f51-9b0f-0da5c50ef783 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:20.138177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9c035-7f06-4f51-9b0f-0da5c50ef783 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:20.138457 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd9c035-7f06-4f51-9b0f-0da5c50ef783 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:20.140034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 165/320] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:29:20.144916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3f058-58d0-4106-8375-4ad7a448863e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:20.145680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3f058-58d0-4106-8375-4ad7a448863e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:20.146108 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e3f058-58d0-4106-8375-4ad7a448863e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:20.152681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 156/321] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:20.166149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-76e3f058-58d0-4106-8375-4ad7a448863e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:20.167549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-76e3f058-58d0-4106-8375-4ad7a448863e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:20.236585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 166/322] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:20.387787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c282d7-bfcf-469c-b15e-8ae4f066914b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:20.388130 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c282d7-bfcf-469c-b15e-8ae4f066914b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:20.388331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c282d7-bfcf-469c-b15e-8ae4f066914b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:20.389842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 157/323] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:20.394749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312b122-c01e-4861-823a-750da69e54ac None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:20.395034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312b122-c01e-4861-823a-750da69e54ac None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:20.396321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6312b122-c01e-4861-823a-750da69e54ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:20.402883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:20.415763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-6312b122-c01e-4861-823a-750da69e54ac None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:20.416965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6312b122-c01e-4861-823a-750da69e54ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:20.425216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c282d7-bfcf-469c-b15e-8ae4f066914b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:20.491143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 167/324] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:20.511441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312b122-c01e-4861-823a-750da69e54ac None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:20.535315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c282d7-bfcf-469c-b15e-8ae4f066914b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:20.557399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:20.572963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cc7a1-2f81-4d97-bf0b-e343fd076ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:20.573811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cc7a1-2f81-4d97-bf0b-e343fd076ad9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:20.574245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cc7a1-2f81-4d97-bf0b-e343fd076ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:20.574519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586cc7a1-2f81-4d97-bf0b-e343fd076ad9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:20.586135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586cc7a1-2f81-4d97-bf0b-e343fd076ad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-586cc7a1-2f81-4d97-bf0b-e343fd076ad9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:20.591010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:20.591886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:20.592921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 158/325] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:20.597758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cc7a1-2f81-4d97-bf0b-e343fd076ad9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:20.631999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6312b122-c01e-4861-823a-750da69e54ac None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:20.666591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:20.674066 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cef385-2526-4a08-bd90-d042b62da2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:20.674702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cef385-2526-4a08-bd90-d042b62da2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:20.675041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cef385-2526-4a08-bd90-d042b62da2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:20.675322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cef385-2526-4a08-bd90-d042b62da2d7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:20.676444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cef385-2526-4a08-bd90-d042b62da2d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:20.678351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cef385-2526-4a08-bd90-d042b62da2d7 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:20.680830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 168/326] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:20.686303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cef385-2526-4a08-bd90-d042b62da2d7 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:20.708402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586cc7a1-2f81-4d97-bf0b-e343fd076ad9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:20.751320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34ea2-8878-4828-98ee-745d545f94c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:20.751927 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34ea2-8878-4828-98ee-745d545f94c9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:20.752206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34ea2-8878-4828-98ee-745d545f94c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:20.752476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c34ea2-8878-4828-98ee-745d545f94c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:20.753171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c34ea2-8878-4828-98ee-745d545f94c9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:20.753476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c34ea2-8878-4828-98ee-745d545f94c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:20.755057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c34ea2-8878-4828-98ee-745d545f94c9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:20.763980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 159/327] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:20.764870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34ea2-8878-4828-98ee-745d545f94c9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:20.789015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cef385-2526-4a08-bd90-d042b62da2d7 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:20.807283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:20.822791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5e715-dd95-488a-a57e-686617d3e4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:20.823384 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5e715-dd95-488a-a57e-686617d3e4cc None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:20.823655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5e715-dd95-488a-a57e-686617d3e4cc None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:20.823919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c5e715-dd95-488a-a57e-686617d3e4cc None admin] PATH_INFO: `/v3/groups` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:20.824528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5e715-dd95-488a-a57e-686617d3e4cc None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:20.825984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c5e715-dd95-488a-a57e-686617d3e4cc None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:20.844808 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 169/328] 10.222.0.39 () {60 vars in 1134 bytes} [Tue Sep 6 22:29:20 2022] POST /identity/v3/groups => generated 226 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:29:20.845821 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5e715-dd95-488a-a57e-686617d3e4cc None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:20.879601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c34ea2-8878-4828-98ee-745d545f94c9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:20.916297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:20.923306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a411a5-6c17-4f6e-860a-5020b331add8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:20.924521 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a411a5-6c17-4f6e-860a-5020b331add8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:20.925018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a411a5-6c17-4f6e-860a-5020b331add8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:20.925520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a411a5-6c17-4f6e-860a-5020b331add8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:20.938526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a411a5-6c17-4f6e-860a-5020b331add8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-b5a411a5-6c17-4f6e-860a-5020b331add8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:20.944890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:20.946062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:20.947040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:20.949185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 160/329] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:20.954918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a411a5-6c17-4f6e-860a-5020b331add8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:20.964959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c5e715-dd95-488a-a57e-686617d3e4cc None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:21.007494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5a7b2-d714-4553-92e2-cc57f1db857d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:21.008185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5a7b2-d714-4553-92e2-cc57f1db857d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:21.008463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5a7b2-d714-4553-92e2-cc57f1db857d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:21.008729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5a7b2-d714-4553-92e2-cc57f1db857d None admin] PATH_INFO: `/v3/users/e832d8c6868945b4916a1f28962f63c4` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:21.009923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5a7b2-d714-4553-92e2-cc57f1db857d None admin] RBAC: Authorizing `identity:get_user(user_id=e832d8c6868945b4916a1f28962f63c4)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:21.011368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5a7b2-d714-4553-92e2-cc57f1db857d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:21.013178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 170/330] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:20 2022] GET /identity/v3/users/e832d8c6868945b4916a1f28962f63c4 => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:21.018605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5a7b2-d714-4553-92e2-cc57f1db857d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:21.077379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a411a5-6c17-4f6e-860a-5020b331add8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:21.079861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:21.127657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959be1ee-5ab3-4267-ac7b-8e317cd167d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:21.128277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959be1ee-5ab3-4267-ac7b-8e317cd167d9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:21.128546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959be1ee-5ab3-4267-ac7b-8e317cd167d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:21.128836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959be1ee-5ab3-4267-ac7b-8e317cd167d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:21.129498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959be1ee-5ab3-4267-ac7b-8e317cd167d9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:21.130851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959be1ee-5ab3-4267-ac7b-8e317cd167d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:21.132841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959be1ee-5ab3-4267-ac7b-8e317cd167d9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:21.144746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 161/331] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:21.145636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5a7b2-d714-4553-92e2-cc57f1db857d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:21.151244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959be1ee-5ab3-4267-ac7b-8e317cd167d9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:21.178691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:21.186964 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d580f4-dcd0-4156-92e2-8fcd759fae32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:21.187669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d580f4-dcd0-4156-92e2-8fcd759fae32 None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:21.187946 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d580f4-dcd0-4156-92e2-8fcd759fae32 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:21.188215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d580f4-dcd0-4156-92e2-8fcd759fae32 None admin] PATH_INFO: `/v3/system/users/e832d8c6868945b4916a1f28962f63c4/roles/4f87bf3cce2d4008a95c1da7369d74bf` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:21.189762 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d580f4-dcd0-4156-92e2-8fcd759fae32 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=e832d8c6868945b4916a1f28962f63c4, role_id=4f87bf3cce2d4008a95c1da7369d74bf)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:21.191164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d580f4-dcd0-4156-92e2-8fcd759fae32 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:21.203534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 171/332] 10.222.0.39 () {58 vars in 1334 bytes} [Tue Sep 6 22:29:21 2022] PUT /identity/v3/system/users/e832d8c6868945b4916a1f28962f63c4/roles/4f87bf3cce2d4008a95c1da7369d74bf => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:21.279289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959be1ee-5ab3-4267-ac7b-8e317cd167d9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:21.321109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6be4ca-11f5-416c-885c-e4153a195c9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:21.321764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6be4ca-11f5-416c-885c-e4153a195c9b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:21.322039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6be4ca-11f5-416c-885c-e4153a195c9b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:21.322308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6be4ca-11f5-416c-885c-e4153a195c9b None admin] PATH_INFO: `/v3/users/a3397e6a4cc5458e9262e07d4fffead8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:21.323566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6be4ca-11f5-416c-885c-e4153a195c9b None admin] RBAC: Authorizing `identity:get_user(user_id=a3397e6a4cc5458e9262e07d4fffead8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:21.325088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6be4ca-11f5-416c-885c-e4153a195c9b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:21.327158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 162/333] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:21 2022] GET /identity/v3/users/a3397e6a4cc5458e9262e07d4fffead8 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:21.333525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d580f4-dcd0-4156-92e2-8fcd759fae32 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:21.451534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d580f4-dcd0-4156-92e2-8fcd759fae32 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:21.486972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff13315-5dc2-4688-8d12-eeac06e041d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:21.487575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff13315-5dc2-4688-8d12-eeac06e041d7 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:21.487852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff13315-5dc2-4688-8d12-eeac06e041d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:21.488123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff13315-5dc2-4688-8d12-eeac06e041d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:21.488871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff13315-5dc2-4688-8d12-eeac06e041d7 None admin] RBAC: Adding query filter params (role.id=aac9ca64da09463ba9b3966c7188d7ee, user.id=a3397e6a4cc5458e9262e07d4fffead8, scope.system=all) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:21.490144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff13315-5dc2-4688-8d12-eeac06e041d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:21.491588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff13315-5dc2-4688-8d12-eeac06e041d7 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:21.505558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 172/334] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Sep 6 22:29:21 2022] GET /identity/v3/role_assignments?user.id=a3397e6a4cc5458e9262e07d4fffead8&scope.system=all&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:23.489538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c3e57-9d0b-413b-a663-d1665c760a00 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:23.490505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c3e57-9d0b-413b-a663-d1665c760a00 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:23.490923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c3e57-9d0b-413b-a663-d1665c760a00 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:23.492668 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 163/335] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:23.498386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d2929-cc2c-4bfc-b0b5-314e47f01476 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:23.499149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d2929-cc2c-4bfc-b0b5-314e47f01476 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:23.499604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629d2929-cc2c-4bfc-b0b5-314e47f01476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:23.521093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-629d2929-cc2c-4bfc-b0b5-314e47f01476 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:23.522503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-629d2929-cc2c-4bfc-b0b5-314e47f01476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:23.573806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:23.596696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 173/336] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:23.897238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c3e57-9d0b-413b-a663-d1665c760a00 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:23.932282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:23.936594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0de4b-351d-4cb7-8cfb-0e704c5d8831 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:23.937170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0de4b-351d-4cb7-8cfb-0e704c5d8831 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:23.937647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d0de4b-351d-4cb7-8cfb-0e704c5d8831 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:23.939614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 174/337] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:23.950265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f247f6-60a2-4cf0-aca4-5c6a04536ccb None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:23.950847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f247f6-60a2-4cf0-aca4-5c6a04536ccb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:23.951319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f247f6-60a2-4cf0-aca4-5c6a04536ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:23.981249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-60f247f6-60a2-4cf0-aca4-5c6a04536ccb None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:23.982778 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-60f247f6-60a2-4cf0-aca4-5c6a04536ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:24.018904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c3e57-9d0b-413b-a663-d1665c760a00 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:24.064618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c025df-eee6-4bd7-94da-42a993f8e0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:24.065745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c025df-eee6-4bd7-94da-42a993f8e0cd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:24.066172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c025df-eee6-4bd7-94da-42a993f8e0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:24.066603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c025df-eee6-4bd7-94da-42a993f8e0cd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:24.079563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c025df-eee6-4bd7-94da-42a993f8e0cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-29c025df-eee6-4bd7-94da-42a993f8e0cd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:24.085285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:24.086418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:24.087456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:24.090053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 164/338] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:24.091429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe61cc9-5a1b-4d56-8991-894d91c47ae1 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:24.092354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe61cc9-5a1b-4d56-8991-894d91c47ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:24.097005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe61cc9-5a1b-4d56-8991-894d91c47ae1 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:24.099834 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 165/339] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:24.101914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe61cc9-5a1b-4d56-8991-894d91c47ae1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:24.130963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 175/340] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:24.132338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9a058-9e26-4360-a6e1-628724cd7ea6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:24.132697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9a058-9e26-4360-a6e1-628724cd7ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:24.133064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9a058-9e26-4360-a6e1-628724cd7ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:24.169522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-6ef9a058-9e26-4360-a6e1-628724cd7ea6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:24.172547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6ef9a058-9e26-4360-a6e1-628724cd7ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:24.258680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 176/341] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:24.265272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:24.277534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe61cc9-5a1b-4d56-8991-894d91c47ae1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:24.314088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ff140-cc37-44f6-be76-62871950b4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:24.314692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ff140-cc37-44f6-be76-62871950b4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:24.315016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ff140-cc37-44f6-be76-62871950b4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:24.315293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2ff140-cc37-44f6-be76-62871950b4c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:24.315953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ff140-cc37-44f6-be76-62871950b4c4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:24.316249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ff140-cc37-44f6-be76-62871950b4c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:24.317736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2ff140-cc37-44f6-be76-62871950b4c4 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:24.326445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 166/342] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:24.332234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9a058-9e26-4360-a6e1-628724cd7ea6 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:24.393119 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:24.460988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9a058-9e26-4360-a6e1-628724cd7ea6 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:24.497787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006f8a2-63aa-41db-9877-f211c2c08c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:24.498401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f8a2-63aa-41db-9877-f211c2c08c76 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:24.498932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f8a2-63aa-41db-9877-f211c2c08c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:24.498932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e006f8a2-63aa-41db-9877-f211c2c08c76 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:24.575174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006f8a2-63aa-41db-9877-f211c2c08c76 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:24.580791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e006f8a2-63aa-41db-9877-f211c2c08c76 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:24.622179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:24.687276 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 177/343] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:24.693457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ff140-cc37-44f6-be76-62871950b4c4 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:24.776555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006f8a2-63aa-41db-9877-f211c2c08c76 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:24.847646 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2ff140-cc37-44f6-be76-62871950b4c4 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:24.899022 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7293529-2ba8-4a23-9f8e-240231435793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:24.899659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7293529-2ba8-4a23-9f8e-240231435793 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:24.899931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7293529-2ba8-4a23-9f8e-240231435793 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:24.900199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7293529-2ba8-4a23-9f8e-240231435793 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:24.901670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7293529-2ba8-4a23-9f8e-240231435793 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:24.903299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7293529-2ba8-4a23-9f8e-240231435793 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:24.905363 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 167/344] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:24 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:24.906226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7293529-2ba8-4a23-9f8e-240231435793 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:24.915204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e006f8a2-63aa-41db-9877-f211c2c08c76 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:24.931358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:24.956065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57231f-8c9c-4bc6-ac69-9513a5007929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:24.956725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57231f-8c9c-4bc6-ac69-9513a5007929 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:24.957044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57231f-8c9c-4bc6-ac69-9513a5007929 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:24.957361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57231f-8c9c-4bc6-ac69-9513a5007929 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:24.965643 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:24.968705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57231f-8c9c-4bc6-ac69-9513a5007929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-1c57231f-8c9c-4bc6-ac69-9513a5007929 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:24.974868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:24.975809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 06 22:29:24.976593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:24.977765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 178/345] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:24.983084 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1caa4-4945-403c-b88f-cb5d37be9122 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:24.983369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1caa4-4945-403c-b88f-cb5d37be9122 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:24.983648 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d1caa4-4945-403c-b88f-cb5d37be9122 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:24.985400 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 179/346] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:24 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:24.986177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1caa4-4945-403c-b88f-cb5d37be9122 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:25.024344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7293529-2ba8-4a23-9f8e-240231435793 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:25.061685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c715c4-4688-4922-bff1-7c565c355c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:25.062246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c715c4-4688-4922-bff1-7c565c355c80 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:25.062525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c715c4-4688-4922-bff1-7c565c355c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:25.062793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c715c4-4688-4922-bff1-7c565c355c80 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:25.075617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c715c4-4688-4922-bff1-7c565c355c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-c4c715c4-4688-4922-bff1-7c565c355c80 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:25.081227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:25.082117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 168/347] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:29:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:25.084861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c715c4-4688-4922-bff1-7c565c355c80 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:25.104348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d1caa4-4945-403c-b88f-cb5d37be9122 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:25.143303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e876aa-ef4a-49eb-9be8-92f489db1d3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:25.143989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e876aa-ef4a-49eb-9be8-92f489db1d3c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:25.144269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e876aa-ef4a-49eb-9be8-92f489db1d3c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:25.144541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e876aa-ef4a-49eb-9be8-92f489db1d3c None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:25.152031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e876aa-ef4a-49eb-9be8-92f489db1d3c None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:25.153587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e876aa-ef4a-49eb-9be8-92f489db1d3c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:25.160302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 180/348] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:24 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:25.161689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c92710-89a1-43f2-a29f-a6b1da4f708d None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:25.161973 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c92710-89a1-43f2-a29f-a6b1da4f708d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:25.162244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c92710-89a1-43f2-a29f-a6b1da4f708d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:25.181123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:25.181342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-a6c92710-89a1-43f2-a29f-a6b1da4f708d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:25.182543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a6c92710-89a1-43f2-a29f-a6b1da4f708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:25.195539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c715c4-4688-4922-bff1-7c565c355c80 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:25.239437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ba1ff-78f9-4480-ab05-74a50a971619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:25.240452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ba1ff-78f9-4480-ab05-74a50a971619 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:25.240638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ba1ff-78f9-4480-ab05-74a50a971619 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:25.240905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86ba1ff-78f9-4480-ab05-74a50a971619 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:25.241576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ba1ff-78f9-4480-ab05-74a50a971619 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:25.241905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ba1ff-78f9-4480-ab05-74a50a971619 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:25.244536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86ba1ff-78f9-4480-ab05-74a50a971619 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:25.256702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:25.258690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 169/349] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:25.259530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ba1ff-78f9-4480-ab05-74a50a971619 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:25.272056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 181/350] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:25.273039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c92710-89a1-43f2-a29f-a6b1da4f708d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:25.378694 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86ba1ff-78f9-4480-ab05-74a50a971619 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:25.403650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c92710-89a1-43f2-a29f-a6b1da4f708d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:25.422855 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8248-21ba-4cbd-942f-447ae89493bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:25.423951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8248-21ba-4cbd-942f-447ae89493bb None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:25.424829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8248-21ba-4cbd-942f-447ae89493bb None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:25.425576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8248-21ba-4cbd-942f-447ae89493bb None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:25.427063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e8248-21ba-4cbd-942f-447ae89493bb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:25.427579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e8248-21ba-4cbd-942f-447ae89493bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:25.429225 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041e8248-21ba-4cbd-942f-447ae89493bb None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:25.440682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 170/351] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:25.441871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8248-21ba-4cbd-942f-447ae89493bb None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:25.448906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:25.463009 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dc52c1-646a-490c-bdc7-4ec95be28447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:25.463619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc52c1-646a-490c-bdc7-4ec95be28447 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:25.463902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc52c1-646a-490c-bdc7-4ec95be28447 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:25.464172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dc52c1-646a-490c-bdc7-4ec95be28447 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:25.464920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc52c1-646a-490c-bdc7-4ec95be28447 None admin] RBAC: Adding query filter params (group.id=e0dfdbe067844c5788d85b1e7040bd19, role.id=aac9ca64da09463ba9b3966c7188d7ee, scope.project.id=a6cc40a4154043ada48161d66116e3c8) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:25.465270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc52c1-646a-490c-bdc7-4ec95be28447 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:25.466657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dc52c1-646a-490c-bdc7-4ec95be28447 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:25.472880 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 182/352] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Sep 6 22:29:25 2022] GET /identity/v3/role_assignments?group.id=e0dfdbe067844c5788d85b1e7040bd19&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 260 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:25.474561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d108dc9-31ea-4619-a954-61beb4afcdfc None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:25.474838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d108dc9-31ea-4619-a954-61beb4afcdfc None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:25.475114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d108dc9-31ea-4619-a954-61beb4afcdfc None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:25.481361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 183/353] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:25 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:25.482230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d108dc9-31ea-4619-a954-61beb4afcdfc None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:25.565300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8248-21ba-4cbd-942f-447ae89493bb None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:25.596403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:25.616485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4770c8-a696-4e69-a1cd-a4b46400c860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:25.616692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4770c8-a696-4e69-a1cd-a4b46400c860 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:25.617967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4770c8-a696-4e69-a1cd-a4b46400c860 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:25.618249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4770c8-a696-4e69-a1cd-a4b46400c860 None admin] PATH_INFO: `/v3/users/e832d8c6868945b4916a1f28962f63c4` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:25.619587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4770c8-a696-4e69-a1cd-a4b46400c860 None admin] RBAC: Authorizing `identity:get_user(user_id=e832d8c6868945b4916a1f28962f63c4)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:25.621072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4770c8-a696-4e69-a1cd-a4b46400c860 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:25.623221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 171/354] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:29:25 2022] GET /identity/v3/users/e832d8c6868945b4916a1f28962f63c4 => generated 293 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:25.624531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78067c-07a5-4275-8437-b08ca5a675ef None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:25.624860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78067c-07a5-4275-8437-b08ca5a675ef None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:25.625174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d78067c-07a5-4275-8437-b08ca5a675ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:25.648752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3d78067c-07a5-4275-8437-b08ca5a675ef None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:25.650015 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3d78067c-07a5-4275-8437-b08ca5a675ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:25.660567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d108dc9-31ea-4619-a954-61beb4afcdfc None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:25.704966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3418fe4-6074-40a4-8eda-43621748629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:25.705961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3418fe4-6074-40a4-8eda-43621748629a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:25.706453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3418fe4-6074-40a4-8eda-43621748629a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:25.706928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3418fe4-6074-40a4-8eda-43621748629a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:25.731711 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 172/355] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:25.732625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78067c-07a5-4275-8437-b08ca5a675ef None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:25.735343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:25.816793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3418fe4-6074-40a4-8eda-43621748629a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:25.819365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3418fe4-6074-40a4-8eda-43621748629a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:25.847571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d78067c-07a5-4275-8437-b08ca5a675ef None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:25.886955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6e0ff-2a44-4444-9db1-0df36e80cbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:25.887584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6e0ff-2a44-4444-9db1-0df36e80cbcb None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:25.887851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6e0ff-2a44-4444-9db1-0df36e80cbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:25.888211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc6e0ff-2a44-4444-9db1-0df36e80cbcb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:25.891404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:25.902628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc6e0ff-2a44-4444-9db1-0df36e80cbcb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-8bc6e0ff-2a44-4444-9db1-0df36e80cbcb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:25.907749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:25.908630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:25.909667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 173/356] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:25.910326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6e0ff-2a44-4444-9db1-0df36e80cbcb None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:25.939610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 184/357] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:25.940252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3418fe4-6074-40a4-8eda-43621748629a None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:25.983926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:26.026875 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc6e0ff-2a44-4444-9db1-0df36e80cbcb None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:26.051932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3418fe4-6074-40a4-8eda-43621748629a None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:26.065033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c15567-3c7b-4204-8344-d9d5647c9dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:26.065647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c15567-3c7b-4204-8344-d9d5647c9dec None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:26.065916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c15567-3c7b-4204-8344-d9d5647c9dec None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:26.066282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c15567-3c7b-4204-8344-d9d5647c9dec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:26.067190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c15567-3c7b-4204-8344-d9d5647c9dec None admin] RBAC: Adding query filter params (role.id=4f87bf3cce2d4008a95c1da7369d74bf, user.id=e832d8c6868945b4916a1f28962f63c4, scope.system=all) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:26.067907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c15567-3c7b-4204-8344-d9d5647c9dec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:26.068950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c15567-3c7b-4204-8344-d9d5647c9dec None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:26.083493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 174/358] 10.222.0.39 () {56 vars in 1308 bytes} [Tue Sep 6 22:29:25 2022] GET /identity/v3/role_assignments?user.id=e832d8c6868945b4916a1f28962f63c4&scope.system=all&role.id=4f87bf3cce2d4008a95c1da7369d74bf => generated 510 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:26.084109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c15567-3c7b-4204-8344-d9d5647c9dec None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:26.101628 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f09d2b-2f60-47bf-adcb-f9f85b17706d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:26.102258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f09d2b-2f60-47bf-adcb-f9f85b17706d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:26.102536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f09d2b-2f60-47bf-adcb-f9f85b17706d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:26.102806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f09d2b-2f60-47bf-adcb-f9f85b17706d None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:26.103487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f09d2b-2f60-47bf-adcb-f9f85b17706d None admin] RBAC: Adding query filter params (name=member) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:26.103786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f09d2b-2f60-47bf-adcb-f9f85b17706d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:26.105199 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f09d2b-2f60-47bf-adcb-f9f85b17706d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:26.115191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 185/359] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:26.123221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f09d2b-2f60-47bf-adcb-f9f85b17706d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:26.181611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:26.201965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c15567-3c7b-4204-8344-d9d5647c9dec None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:26.229345 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:26.253268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8b4f6-3d2a-40e9-a216-be3a4b663d7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:26.254173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8b4f6-3d2a-40e9-a216-be3a4b663d7a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:26.255858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8b4f6-3d2a-40e9-a216-be3a4b663d7a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:26.255858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f8b4f6-3d2a-40e9-a216-be3a4b663d7a None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:26.256712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8b4f6-3d2a-40e9-a216-be3a4b663d7a None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:26.261013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f8b4f6-3d2a-40e9-a216-be3a4b663d7a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:26.263103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 175/360] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:26 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:26.264072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8b4f6-3d2a-40e9-a216-be3a4b663d7a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:26.327709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f09d2b-2f60-47bf-adcb-f9f85b17706d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:26.374158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079b3d28-c3ac-4cc1-b6fa-fe62b1281db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:26.374852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b3d28-c3ac-4cc1-b6fa-fe62b1281db0 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:26.375045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b3d28-c3ac-4cc1-b6fa-fe62b1281db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:26.375372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079b3d28-c3ac-4cc1-b6fa-fe62b1281db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:26.446072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f8b4f6-3d2a-40e9-a216-be3a4b663d7a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:26.469500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b3d28-c3ac-4cc1-b6fa-fe62b1281db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:26.471096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079b3d28-c3ac-4cc1-b6fa-fe62b1281db0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:26.485278 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a2755-ea6f-4575-89bf-1c5a266d264a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:26.486141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a2755-ea6f-4575-89bf-1c5a266d264a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:26.486417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a2755-ea6f-4575-89bf-1c5a266d264a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:26.486685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066a2755-ea6f-4575-89bf-1c5a266d264a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:26.497966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:26.501749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066a2755-ea6f-4575-89bf-1c5a266d264a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-066a2755-ea6f-4575-89bf-1c5a266d264a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:26.508188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:26.509204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:26.509998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 176/361] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:29:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:26.510651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a2755-ea6f-4575-89bf-1c5a266d264a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:26.585722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:26.624451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 186/362] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 502 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:26.626687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72465de-42b8-4ea9-a40c-57df80ceeb9d None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:26.627353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72465de-42b8-4ea9-a40c-57df80ceeb9d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:26.627353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72465de-42b8-4ea9-a40c-57df80ceeb9d None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:26.629537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 187/363] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:26.630489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72465de-42b8-4ea9-a40c-57df80ceeb9d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:26.663176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066a2755-ea6f-4575-89bf-1c5a266d264a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:26.702640 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a8c0f-c06b-4b29-bbd1-b41e1a879c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:26.703348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8c0f-c06b-4b29-bbd1-b41e1a879c95 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:26.703662 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8c0f-c06b-4b29-bbd1-b41e1a879c95 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:26.703978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5a8c0f-c06b-4b29-bbd1-b41e1a879c95 None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:26.705158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a8c0f-c06b-4b29-bbd1-b41e1a879c95 None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:26.706664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5a8c0f-c06b-4b29-bbd1-b41e1a879c95 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:26.708804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 177/364] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:26 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:26.709589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a8c0f-c06b-4b29-bbd1-b41e1a879c95 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:26.767059 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72465de-42b8-4ea9-a40c-57df80ceeb9d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:26.790029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:26.811687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f49cd3-fb15-4739-8259-7e0e92a28bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:26.812585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f49cd3-fb15-4739-8259-7e0e92a28bbf None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:26.812896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f49cd3-fb15-4739-8259-7e0e92a28bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:26.813222 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f49cd3-fb15-4739-8259-7e0e92a28bbf None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:26.813918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f49cd3-fb15-4739-8259-7e0e92a28bbf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:26.814221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f49cd3-fb15-4739-8259-7e0e92a28bbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:26.815937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f49cd3-fb15-4739-8259-7e0e92a28bbf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:26.832015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 188/365] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:26.835206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ff8cf-19c3-4f18-ba44-30c0c4705ff1 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:26.835481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ff8cf-19c3-4f18-ba44-30c0c4705ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:26.835747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11ff8cf-19c3-4f18-ba44-30c0c4705ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:26.851340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5a8c0f-c06b-4b29-bbd1-b41e1a879c95 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:26.855061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-f11ff8cf-19c3-4f18-ba44-30c0c4705ff1 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:26.855863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f11ff8cf-19c3-4f18-ba44-30c0c4705ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:26.870470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:26.892089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb709a-2289-4b0d-a114-b862bd6979d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:26.892888 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb709a-2289-4b0d-a114-b862bd6979d4 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:26.893243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb709a-2289-4b0d-a114-b862bd6979d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:26.893588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-decb709a-2289-4b0d-a114-b862bd6979d4 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:26.894811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb709a-2289-4b0d-a114-b862bd6979d4 None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:26.896495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-decb709a-2289-4b0d-a114-b862bd6979d4 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:26.898349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 178/366] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:26 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:26.899238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb709a-2289-4b0d-a114-b862bd6979d4 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:26.953670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 189/367] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:26.954487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ff8cf-19c3-4f18-ba44-30c0c4705ff1 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:27.058994 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-decb709a-2289-4b0d-a114-b862bd6979d4 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:27.079896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11ff8cf-19c3-4f18-ba44-30c0c4705ff1 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:27.098637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:27.101618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602b67f-4bee-4c2a-840d-bd265ced8de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:27.102238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602b67f-4bee-4c2a-840d-bd265ced8de9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:27.102516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602b67f-4bee-4c2a-840d-bd265ced8de9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:27.102788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602b67f-4bee-4c2a-840d-bd265ced8de9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:27.103582 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602b67f-4bee-4c2a-840d-bd265ced8de9 None admin] RBAC: Adding query filter params (group.id=e0dfdbe067844c5788d85b1e7040bd19, role.id=952084593c184a78a952e1797a0b6c6b, scope.project.id=a6cc40a4154043ada48161d66116e3c8) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:27.103895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602b67f-4bee-4c2a-840d-bd265ced8de9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:27.105793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7602b67f-4bee-4c2a-840d-bd265ced8de9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:27.112204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 179/368] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Sep 6 22:29:26 2022] GET /identity/v3/role_assignments?group.id=e0dfdbe067844c5788d85b1e7040bd19&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=952084593c184a78a952e1797a0b6c6b => generated 260 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:27.113045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602b67f-4bee-4c2a-840d-bd265ced8de9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:27.131324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd3ca6-2af4-49b8-bde0-83544508f4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:27.131946 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd3ca6-2af4-49b8-bde0-83544508f4cd None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:27.133020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd3ca6-2af4-49b8-bde0-83544508f4cd None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:27.133303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccd3ca6-2af4-49b8-bde0-83544508f4cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:27.199845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:27.231977 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd3ca6-2af4-49b8-bde0-83544508f4cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:27.233722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccd3ca6-2af4-49b8-bde0-83544508f4cd None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:27.299445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602b67f-4bee-4c2a-840d-bd265ced8de9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:27.339176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fb19c-6865-43ff-9293-5fe2efe7f24d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:27.340203 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fb19c-6865-43ff-9293-5fe2efe7f24d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:27.340675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fb19c-6865-43ff-9293-5fe2efe7f24d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:27.341186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038fb19c-6865-43ff-9293-5fe2efe7f24d None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:27.342577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fb19c-6865-43ff-9293-5fe2efe7f24d None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:27.344266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038fb19c-6865-43ff-9293-5fe2efe7f24d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:27.346609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 180/369] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:27 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:27.347772 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fb19c-6865-43ff-9293-5fe2efe7f24d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:27.370276 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 190/370] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:27.371137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd3ca6-2af4-49b8-bde0-83544508f4cd None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:27.390612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:27.464363 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038fb19c-6865-43ff-9293-5fe2efe7f24d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:27.482379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:27.492316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccd3ca6-2af4-49b8-bde0-83544508f4cd None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:27.502437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d774ad6-fe5b-476e-ab05-efbc2f07cf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:27.503525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d774ad6-fe5b-476e-ab05-efbc2f07cf5f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:27.504007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d774ad6-fe5b-476e-ab05-efbc2f07cf5f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:27.505049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d774ad6-fe5b-476e-ab05-efbc2f07cf5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:27.519325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d774ad6-fe5b-476e-ab05-efbc2f07cf5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2d774ad6-fe5b-476e-ab05-efbc2f07cf5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:27.524985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:27.525897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 181/371] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:29:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:27.531239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d774ad6-fe5b-476e-ab05-efbc2f07cf5f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:27.542622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef3bd0-37b6-4012-8b0e-60ea1e56acb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:27.543577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef3bd0-37b6-4012-8b0e-60ea1e56acb6 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:27.544041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef3bd0-37b6-4012-8b0e-60ea1e56acb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:27.544593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ef3bd0-37b6-4012-8b0e-60ea1e56acb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:27.545625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef3bd0-37b6-4012-8b0e-60ea1e56acb6 None admin] RBAC: Adding query filter params (group.id=e0dfdbe067844c5788d85b1e7040bd19, role.id=aac9ca64da09463ba9b3966c7188d7ee, scope.project.id=ef03f831e4b94427a2be5482f556af22) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:27.546176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef3bd0-37b6-4012-8b0e-60ea1e56acb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:27.548656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ef3bd0-37b6-4012-8b0e-60ea1e56acb6 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:27.555812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 191/372] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Sep 6 22:29:27 2022] GET /identity/v3/role_assignments?group.id=e0dfdbe067844c5788d85b1e7040bd19&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:27.557053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef3bd0-37b6-4012-8b0e-60ea1e56acb6 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:27.643996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d774ad6-fe5b-476e-ab05-efbc2f07cf5f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:27.651280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:27.680693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ef3bd0-37b6-4012-8b0e-60ea1e56acb6 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:27.682883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8299d6c-b063-4ccb-b331-d3a97ce7612f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:27.683528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8299d6c-b063-4ccb-b331-d3a97ce7612f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:27.683810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8299d6c-b063-4ccb-b331-d3a97ce7612f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:27.684081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8299d6c-b063-4ccb-b331-d3a97ce7612f None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:27.685277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8299d6c-b063-4ccb-b331-d3a97ce7612f None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:27.686771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8299d6c-b063-4ccb-b331-d3a97ce7612f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:27.688716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 182/373] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:27 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:27.694200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8299d6c-b063-4ccb-b331-d3a97ce7612f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:27.721535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecb779-ca0d-496c-8608-8f083a7e0aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:27.722152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecb779-ca0d-496c-8608-8f083a7e0aee None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:27.722542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecb779-ca0d-496c-8608-8f083a7e0aee None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:27.722868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ecb779-ca0d-496c-8608-8f083a7e0aee None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:27.724420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecb779-ca0d-496c-8608-8f083a7e0aee None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:27.724966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecb779-ca0d-496c-8608-8f083a7e0aee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:27.726456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ecb779-ca0d-496c-8608-8f083a7e0aee None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:27.741271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 192/374] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:27.743630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecb779-ca0d-496c-8608-8f083a7e0aee None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:27.744478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:27.807453 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8299d6c-b063-4ccb-b331-d3a97ce7612f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:27.843601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884b1e4-6eca-4086-b4ec-af511b28f36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:27.844245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884b1e4-6eca-4086-b4ec-af511b28f36a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:27.844520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884b1e4-6eca-4086-b4ec-af511b28f36a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:27.844811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a884b1e4-6eca-4086-b4ec-af511b28f36a None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:27.845964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884b1e4-6eca-4086-b4ec-af511b28f36a None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:27.847478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a884b1e4-6eca-4086-b4ec-af511b28f36a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:27.850041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 183/375] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:27 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:27.855887 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884b1e4-6eca-4086-b4ec-af511b28f36a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:27.864574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ecb779-ca0d-496c-8608-8f083a7e0aee None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:27.904566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff482c0d-b6f1-4065-a72f-1e233fdef64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:27.905253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff482c0d-b6f1-4065-a72f-1e233fdef64c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:27.905528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff482c0d-b6f1-4065-a72f-1e233fdef64c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:27.905791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff482c0d-b6f1-4065-a72f-1e233fdef64c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:27.916454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:27.967058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a884b1e4-6eca-4086-b4ec-af511b28f36a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:27.981184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:27.986891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff482c0d-b6f1-4065-a72f-1e233fdef64c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:27.988479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff482c0d-b6f1-4065-a72f-1e233fdef64c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:28.006450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da47e71-5059-4b7a-967f-a0f043590eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:28.006991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da47e71-5059-4b7a-967f-a0f043590eb1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:28.007564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da47e71-5059-4b7a-967f-a0f043590eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:28.007842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da47e71-5059-4b7a-967f-a0f043590eb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:28.008581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da47e71-5059-4b7a-967f-a0f043590eb1 None admin] RBAC: Adding query filter params (group.id=e0dfdbe067844c5788d85b1e7040bd19, role.id=952084593c184a78a952e1797a0b6c6b, scope.project.id=ef03f831e4b94427a2be5482f556af22) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:28.008918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da47e71-5059-4b7a-967f-a0f043590eb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:28.010374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da47e71-5059-4b7a-967f-a0f043590eb1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:28.016525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 184/376] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Sep 6 22:29:27 2022] GET /identity/v3/role_assignments?group.id=e0dfdbe067844c5788d85b1e7040bd19&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=952084593c184a78a952e1797a0b6c6b => generated 260 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:28.112899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 193/377] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:28.115394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da47e71-5059-4b7a-967f-a0f043590eb1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:28.231098 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da47e71-5059-4b7a-967f-a0f043590eb1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:28.266899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9241d9-2e24-4521-944f-7e23a6df5e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:28.267448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9241d9-2e24-4521-944f-7e23a6df5e54 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:28.267785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9241d9-2e24-4521-944f-7e23a6df5e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:28.268056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9241d9-2e24-4521-944f-7e23a6df5e54 None admin] PATH_INFO: `/v3/projects/693d7b3554de4d2c9e0e732722fe61d1` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:28.269217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9241d9-2e24-4521-944f-7e23a6df5e54 None admin] RBAC: Authorizing `identity:get_project(project_id=693d7b3554de4d2c9e0e732722fe61d1)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:28.271510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9241d9-2e24-4521-944f-7e23a6df5e54 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:28.276867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 185/378] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:28 2022] GET /identity/v3/projects/693d7b3554de4d2c9e0e732722fe61d1 => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:28.279117 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff482c0d-b6f1-4065-a72f-1e233fdef64c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:28.326817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:28.394279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff482c0d-b6f1-4065-a72f-1e233fdef64c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:28.430516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd05458-e12a-4db7-a412-8476900b38e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:28.431140 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd05458-e12a-4db7-a412-8476900b38e3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:28.431411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd05458-e12a-4db7-a412-8476900b38e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:28.431677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd05458-e12a-4db7-a412-8476900b38e3 None admin] PATH_INFO: `/v3/groups/a4e50171f099483b9b32cb7aaad16eea` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:28.436723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd05458-e12a-4db7-a412-8476900b38e3 None admin] RBAC: Authorizing `identity:get_group(group_id=a4e50171f099483b9b32cb7aaad16eea)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:28.438169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd05458-e12a-4db7-a412-8476900b38e3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:28.440171 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 194/379] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:28 2022] GET /identity/v3/groups/a4e50171f099483b9b32cb7aaad16eea => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:28.446314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9241d9-2e24-4521-944f-7e23a6df5e54 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:28.517332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:28.588428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220aa75-6e1d-4363-8f1e-3fcf6869faa5 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:28.588749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220aa75-6e1d-4363-8f1e-3fcf6869faa5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:28.589010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220aa75-6e1d-4363-8f1e-3fcf6869faa5 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:28.590606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 195/380] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:28.594141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9241d9-2e24-4521-944f-7e23a6df5e54 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:28.595513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60517c-3abf-4c19-8038-696ca7f6423e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:28.595789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60517c-3abf-4c19-8038-696ca7f6423e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:28.596059 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f60517c-3abf-4c19-8038-696ca7f6423e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:28.635022 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18707235-bcad-43e4-877d-23431ee08945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:28.635633 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18707235-bcad-43e4-877d-23431ee08945 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:28.635908 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18707235-bcad-43e4-877d-23431ee08945 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:28.636178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18707235-bcad-43e4-877d-23431ee08945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:28.636928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18707235-bcad-43e4-877d-23431ee08945 None admin] RBAC: Adding query filter params (group.id=a4e50171f099483b9b32cb7aaad16eea, role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=693d7b3554de4d2c9e0e732722fe61d1) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:28.637235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18707235-bcad-43e4-877d-23431ee08945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:28.637794 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2f60517c-3abf-4c19-8038-696ca7f6423e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:28.638660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18707235-bcad-43e4-877d-23431ee08945 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:28.638990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2f60517c-3abf-4c19-8038-696ca7f6423e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:28.644652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 186/381] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Sep 6 22:29:28 2022] GET /identity/v3/role_assignments?group.id=a4e50171f099483b9b32cb7aaad16eea&scope.project.id=693d7b3554de4d2c9e0e732722fe61d1&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:28.742259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 196/382] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:29.219927 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18707235-bcad-43e4-877d-23431ee08945 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:29.349335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18707235-bcad-43e4-877d-23431ee08945 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:29.359190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:29.393713 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa0725-1af5-4680-a2fe-63fc4503987a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:29.394351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa0725-1af5-4680-a2fe-63fc4503987a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:29.394630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa0725-1af5-4680-a2fe-63fc4503987a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:29.394901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baa0725-1af5-4680-a2fe-63fc4503987a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:29.411267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baa0725-1af5-4680-a2fe-63fc4503987a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-1baa0725-1af5-4680-a2fe-63fc4503987a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:29.416690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:29.417611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:29.418432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:29.420905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 187/383] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:29.425841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60517c-3abf-4c19-8038-696ca7f6423e None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:29.477960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:29.569568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f60517c-3abf-4c19-8038-696ca7f6423e None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:29.618472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24097636-c640-4a12-b936-4de62c865c66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:29.619085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24097636-c640-4a12-b936-4de62c865c66 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:29.619360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24097636-c640-4a12-b936-4de62c865c66 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:29.619630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24097636-c640-4a12-b936-4de62c865c66 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:29.620298 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24097636-c640-4a12-b936-4de62c865c66 None admin] RBAC: Adding query filter params (name=member) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:29.620599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24097636-c640-4a12-b936-4de62c865c66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:29.621993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24097636-c640-4a12-b936-4de62c865c66 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:29.630731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 197/384] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:29.636708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa0725-1af5-4680-a2fe-63fc4503987a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:29.745869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baa0725-1af5-4680-a2fe-63fc4503987a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:29.781659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fcda9-e39c-43f7-bc4a-b82967e09f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:29.782293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fcda9-e39c-43f7-bc4a-b82967e09f44 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:29.782571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fcda9-e39c-43f7-bc4a-b82967e09f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:29.782844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60fcda9-e39c-43f7-bc4a-b82967e09f44 None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:29.783959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fcda9-e39c-43f7-bc4a-b82967e09f44 None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:29.785437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60fcda9-e39c-43f7-bc4a-b82967e09f44 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:29.787515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 188/385] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:29 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:29.793032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24097636-c640-4a12-b936-4de62c865c66 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:29.908600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:29.919630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24097636-c640-4a12-b936-4de62c865c66 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:29.958852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82620f6c-0f1c-48bb-a4c7-60ad5f87aac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:29.959554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620f6c-0f1c-48bb-a4c7-60ad5f87aac6 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:29.959839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620f6c-0f1c-48bb-a4c7-60ad5f87aac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:29.960111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82620f6c-0f1c-48bb-a4c7-60ad5f87aac6 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:29.961311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82620f6c-0f1c-48bb-a4c7-60ad5f87aac6 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:29.962747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82620f6c-0f1c-48bb-a4c7-60ad5f87aac6 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:29.969204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 198/386] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:29 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:29.970679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fcda9-e39c-43f7-bc4a-b82967e09f44 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:30.013092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:30.054166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92886050-1242-4c10-9de2-f93bca65c714 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:30.054822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92886050-1242-4c10-9de2-f93bca65c714 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:30.055306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92886050-1242-4c10-9de2-f93bca65c714 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:30.057190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 199/387] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:29:30.062635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73168773-a675-4fb2-95ef-eaf2636d3ddf None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:30.063183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73168773-a675-4fb2-95ef-eaf2636d3ddf None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:30.063653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73168773-a675-4fb2-95ef-eaf2636d3ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:30.088334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60fcda9-e39c-43f7-bc4a-b82967e09f44 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:30.097773 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-73168773-a675-4fb2-95ef-eaf2636d3ddf None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:30.100031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-73168773-a675-4fb2-95ef-eaf2636d3ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:30.133527 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba5c42-7a3a-4ff4-b192-c3067b01d13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:30.134204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba5c42-7a3a-4ff4-b192-c3067b01d13d None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:30.134484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba5c42-7a3a-4ff4-b192-c3067b01d13d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:30.134753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ba5c42-7a3a-4ff4-b192-c3067b01d13d None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8/groups/e0dfdbe067844c5788d85b1e7040bd19/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:30.136778 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba5c42-7a3a-4ff4-b192-c3067b01d13d None admin] RBAC: Authorizing `identity:create_grant(project_id=a6cc40a4154043ada48161d66116e3c8, group_id=e0dfdbe067844c5788d85b1e7040bd19, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:30.138282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ba5c42-7a3a-4ff4-b192-c3067b01d13d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:30.174822 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 189/388] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Sep 6 22:29:29 2022] PUT /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8/groups/e0dfdbe067844c5788d85b1e7040bd19/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:30.199314 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 200/389] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:30.663191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba5c42-7a3a-4ff4-b192-c3067b01d13d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:30.736518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76739a-d296-4955-bb30-bd6dffb1098b None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:30.736863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76739a-d296-4955-bb30-bd6dffb1098b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:30.737091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd76739a-d296-4955-bb30-bd6dffb1098b None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:30.738717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 201/390] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:29:30.744328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e8773-0917-46a1-b36b-d37febcbe727 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:30.744606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e8773-0917-46a1-b36b-d37febcbe727 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:30.744928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e8773-0917-46a1-b36b-d37febcbe727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:30.769740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-686e8773-0917-46a1-b36b-d37febcbe727 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:30.771488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-686e8773-0917-46a1-b36b-d37febcbe727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:30.777486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:30.786160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ba5c42-7a3a-4ff4-b192-c3067b01d13d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:30.837126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddaf57-3647-4906-adf6-43db6df424a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:30.838380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddaf57-3647-4906-adf6-43db6df424a0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:30.838857 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddaf57-3647-4906-adf6-43db6df424a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:30.839319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ddaf57-3647-4906-adf6-43db6df424a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:30.854466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ddaf57-3647-4906-adf6-43db6df424a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:30.857697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 202/391] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:30.859467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bf6c1-70bf-45bd-8e54-7e2b420fa25e None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:30.859911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bf6c1-70bf-45bd-8e54-7e2b420fa25e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-68ddaf57-3647-4906-adf6-43db6df424a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:30.860209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:30.861129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:30.861919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:30.863172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662bf6c1-70bf-45bd-8e54-7e2b420fa25e None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:30.865600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 203/392] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:30.872781 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 190/393] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:29:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:30.874340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eaf4e5-ddd6-41ed-ba2e-de10d705190c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:30.874839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eaf4e5-ddd6-41ed-ba2e-de10d705190c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:30.875325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26eaf4e5-ddd6-41ed-ba2e-de10d705190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:30.878312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddaf57-3647-4906-adf6-43db6df424a0 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:30.897348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-26eaf4e5-ddd6-41ed-ba2e-de10d705190c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:30.898926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-26eaf4e5-ddd6-41ed-ba2e-de10d705190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:30.922988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:31.015170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 204/394] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:31.034845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ddaf57-3647-4906-adf6-43db6df424a0 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:31.073180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e81a-6666-4814-a030-e7a831377266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:31.074802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e81a-6666-4814-a030-e7a831377266 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:31.074802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e81a-6666-4814-a030-e7a831377266 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:31.074802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e69e81a-6666-4814-a030-e7a831377266 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:31.074802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e81a-6666-4814-a030-e7a831377266 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:31.075073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e81a-6666-4814-a030-e7a831377266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:31.076491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e69e81a-6666-4814-a030-e7a831377266 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:31.085735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 191/395] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:31.091266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eaf4e5-ddd6-41ed-ba2e-de10d705190c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:31.197312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26eaf4e5-ddd6-41ed-ba2e-de10d705190c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:31.238003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abde865-9392-45b4-aa94-36ca816b190f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:31.238668 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abde865-9392-45b4-aa94-36ca816b190f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:31.239447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abde865-9392-45b4-aa94-36ca816b190f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:31.239713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abde865-9392-45b4-aa94-36ca816b190f None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:31.241050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abde865-9392-45b4-aa94-36ca816b190f None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:31.242478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abde865-9392-45b4-aa94-36ca816b190f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:31.244219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 205/396] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:31 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:31.249945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e81a-6666-4814-a030-e7a831377266 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:31.271997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:31.298143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abde865-9392-45b4-aa94-36ca816b190f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:31.338149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:31.396165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e69e81a-6666-4814-a030-e7a831377266 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:31.456338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b65cc-ebfa-42b1-beac-8b15bd2f7b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:31.460907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b65cc-ebfa-42b1-beac-8b15bd2f7b96 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:31.460907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b65cc-ebfa-42b1-beac-8b15bd2f7b96 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:31.460907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0b65cc-ebfa-42b1-beac-8b15bd2f7b96 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:31.460907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b65cc-ebfa-42b1-beac-8b15bd2f7b96 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:31.460907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0b65cc-ebfa-42b1-beac-8b15bd2f7b96 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:31.462095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 192/397] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:31 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:31.467600 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b65cc-ebfa-42b1-beac-8b15bd2f7b96 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:31.478807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abde865-9392-45b4-aa94-36ca816b190f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:31.528582 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad5f61-f955-4495-bde6-694e1d7fcd5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:31.529347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad5f61-f955-4495-bde6-694e1d7fcd5f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:31.529634 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad5f61-f955-4495-bde6-694e1d7fcd5f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:31.529902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad5f61-f955-4495-bde6-694e1d7fcd5f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:31.545534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad5f61-f955-4495-bde6-694e1d7fcd5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-9fad5f61-f955-4495-bde6-694e1d7fcd5f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:31.552847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:31.554364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:31.555156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 206/398] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:29:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:31.555811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad5f61-f955-4495-bde6-694e1d7fcd5f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:31.591820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:31.623421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0b65cc-ebfa-42b1-beac-8b15bd2f7b96 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:31.677978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a7cb-2a59-4a65-9efa-1aab3f901d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:31.679418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a7cb-2a59-4a65-9efa-1aab3f901d55 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:31.679918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a7cb-2a59-4a65-9efa-1aab3f901d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:31.680379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a115a7cb-2a59-4a65-9efa-1aab3f901d55 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8/groups/e0dfdbe067844c5788d85b1e7040bd19/roles/952084593c184a78a952e1797a0b6c6b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:31.683824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115a7cb-2a59-4a65-9efa-1aab3f901d55 None admin] RBAC: Authorizing `identity:create_grant(project_id=a6cc40a4154043ada48161d66116e3c8, group_id=e0dfdbe067844c5788d85b1e7040bd19, role_id=952084593c184a78a952e1797a0b6c6b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:31.686358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a115a7cb-2a59-4a65-9efa-1aab3f901d55 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:31.687583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:31.714933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 193/399] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Sep 6 22:29:31 2022] PUT /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8/groups/e0dfdbe067844c5788d85b1e7040bd19/roles/952084593c184a78a952e1797a0b6c6b => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:31.716228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a7cb-2a59-4a65-9efa-1aab3f901d55 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:31.732303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad5f61-f955-4495-bde6-694e1d7fcd5f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:31.782495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e40c5-d729-4944-b675-1ffbafd52845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:31.783174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e40c5-d729-4944-b675-1ffbafd52845 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:31.783445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e40c5-d729-4944-b675-1ffbafd52845 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:31.783720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e40c5-d729-4944-b675-1ffbafd52845 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:31.799677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e40c5-d729-4944-b675-1ffbafd52845 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-8a6e40c5-d729-4944-b675-1ffbafd52845 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:31.811645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:31.812650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 207/400] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:31.813470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5cb81-60be-45b1-b1dc-a15f55b94ed0 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:31.814200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5cb81-60be-45b1-b1dc-a15f55b94ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:31.814200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5cb81-60be-45b1-b1dc-a15f55b94ed0 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:31.814200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 208/401] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:31.816807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5cb81-60be-45b1-b1dc-a15f55b94ed0 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:31.875806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a115a7cb-2a59-4a65-9efa-1aab3f901d55 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:31.915201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bc59d5-1bc5-4d22-86df-c11fe0056bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:31.915806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc59d5-1bc5-4d22-86df-c11fe0056bbd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:31.916081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc59d5-1bc5-4d22-86df-c11fe0056bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:31.916356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bc59d5-1bc5-4d22-86df-c11fe0056bbd None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:31.917047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc59d5-1bc5-4d22-86df-c11fe0056bbd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:31.917542 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc59d5-1bc5-4d22-86df-c11fe0056bbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:31.918728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bc59d5-1bc5-4d22-86df-c11fe0056bbd None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:31.927864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 194/402] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:31.928945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e69084-c1e3-42ba-80a0-3e74486b7820 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:31.929171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e69084-c1e3-42ba-80a0-3e74486b7820 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:31.929440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e69084-c1e3-42ba-80a0-3e74486b7820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:31.949732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-57e69084-c1e3-42ba-80a0-3e74486b7820 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:31.952706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-57e69084-c1e3-42ba-80a0-3e74486b7820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:31.952866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5cb81-60be-45b1-b1dc-a15f55b94ed0 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:31.953252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:31.995039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0aa00-47ef-40d6-a0a3-5022972874f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:31.995643 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0aa00-47ef-40d6-a0a3-5022972874f0 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:31.995913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0aa00-47ef-40d6-a0a3-5022972874f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:31.996282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc0aa00-47ef-40d6-a0a3-5022972874f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:31.997033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0aa00-47ef-40d6-a0a3-5022972874f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:31.997327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0aa00-47ef-40d6-a0a3-5022972874f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:31.998842 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc0aa00-47ef-40d6-a0a3-5022972874f0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:32.008840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 209/403] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:32.009768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0aa00-47ef-40d6-a0a3-5022972874f0 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:32.012467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:32.026674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 195/404] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:32.027672 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e69084-c1e3-42ba-80a0-3e74486b7820 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:32.122551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc0aa00-47ef-40d6-a0a3-5022972874f0 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:32.159551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21e333-2c1f-49f6-93ac-9dbcc15e2262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:32.160184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21e333-2c1f-49f6-93ac-9dbcc15e2262 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:32.160681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21e333-2c1f-49f6-93ac-9dbcc15e2262 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:32.161131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b21e333-2c1f-49f6-93ac-9dbcc15e2262 None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:32.162350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21e333-2c1f-49f6-93ac-9dbcc15e2262 None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:32.163871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b21e333-2c1f-49f6-93ac-9dbcc15e2262 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:32.165515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 210/405] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:32 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:32.171246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21e333-2c1f-49f6-93ac-9dbcc15e2262 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:32.236343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e69084-c1e3-42ba-80a0-3e74486b7820 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:32.285084 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:32.319202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4f1bf-afdd-423d-bf1a-39b90af1333e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:32.319893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4f1bf-afdd-423d-bf1a-39b90af1333e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:32.320171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4f1bf-afdd-423d-bf1a-39b90af1333e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:32.320440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d4f1bf-afdd-423d-bf1a-39b90af1333e None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:32.321588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4f1bf-afdd-423d-bf1a-39b90af1333e None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:32.324206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b21e333-2c1f-49f6-93ac-9dbcc15e2262 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:32.324341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d4f1bf-afdd-423d-bf1a-39b90af1333e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:32.331843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 196/406] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:32 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:32.333139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4f1bf-afdd-423d-bf1a-39b90af1333e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:32.346282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:32.390625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45b188-710f-458a-b56d-b1cb13d4996c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:32.391276 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45b188-710f-458a-b56d-b1cb13d4996c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:32.391681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45b188-710f-458a-b56d-b1cb13d4996c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:32.392112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f45b188-710f-458a-b56d-b1cb13d4996c None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:32.393325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45b188-710f-458a-b56d-b1cb13d4996c None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:32.396822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f45b188-710f-458a-b56d-b1cb13d4996c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:32.399345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 211/407] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:32 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:32.400146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45b188-710f-458a-b56d-b1cb13d4996c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:32.454343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d4f1bf-afdd-423d-bf1a-39b90af1333e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:32.492215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ae38d-baa0-45ec-abf1-00b623862999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:32.493186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ae38d-baa0-45ec-abf1-00b623862999 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:32.493629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ae38d-baa0-45ec-abf1-00b623862999 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:32.494055 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660ae38d-baa0-45ec-abf1-00b623862999 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:32.495423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ae38d-baa0-45ec-abf1-00b623862999 None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:32.497132 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660ae38d-baa0-45ec-abf1-00b623862999 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:32.499521 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 197/408] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:32 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:32.505193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ae38d-baa0-45ec-abf1-00b623862999 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:32.505333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f45b188-710f-458a-b56d-b1cb13d4996c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:32.541990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db397c9-5c69-40c5-b5ec-fb8f2b5a1550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:32.542677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db397c9-5c69-40c5-b5ec-fb8f2b5a1550 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:32.543017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db397c9-5c69-40c5-b5ec-fb8f2b5a1550 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:32.543216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db397c9-5c69-40c5-b5ec-fb8f2b5a1550 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:32.550441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:32.555431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8db397c9-5c69-40c5-b5ec-fb8f2b5a1550 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-8db397c9-5c69-40c5-b5ec-fb8f2b5a1550 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:32.561110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:32.562187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:32.563075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 212/409] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:29:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:32.563753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db397c9-5c69-40c5-b5ec-fb8f2b5a1550 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:32.604603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:32.644330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660ae38d-baa0-45ec-abf1-00b623862999 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:32.681858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db397c9-5c69-40c5-b5ec-fb8f2b5a1550 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:32.692412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84904fe7-ce3d-4f1a-9036-e4ab00e883a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:32.693726 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84904fe7-ce3d-4f1a-9036-e4ab00e883a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:32.694142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84904fe7-ce3d-4f1a-9036-e4ab00e883a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:32.694553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84904fe7-ce3d-4f1a-9036-e4ab00e883a9 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22/groups/e0dfdbe067844c5788d85b1e7040bd19/roles/952084593c184a78a952e1797a0b6c6b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:32.697306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84904fe7-ce3d-4f1a-9036-e4ab00e883a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef03f831e4b94427a2be5482f556af22, group_id=e0dfdbe067844c5788d85b1e7040bd19, role_id=952084593c184a78a952e1797a0b6c6b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:32.699005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84904fe7-ce3d-4f1a-9036-e4ab00e883a9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:32.715847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 198/410] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Sep 6 22:29:32 2022] PUT /identity/v3/projects/ef03f831e4b94427a2be5482f556af22/groups/e0dfdbe067844c5788d85b1e7040bd19/roles/952084593c184a78a952e1797a0b6c6b => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:32.717105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84904fe7-ce3d-4f1a-9036-e4ab00e883a9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:32.719485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea572ce4-0ac8-4f0e-9383-babf77982752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:32.720169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea572ce4-0ac8-4f0e-9383-babf77982752 None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:32.720446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea572ce4-0ac8-4f0e-9383-babf77982752 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:32.720719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea572ce4-0ac8-4f0e-9383-babf77982752 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22/groups/e0dfdbe067844c5788d85b1e7040bd19/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:32.724667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea572ce4-0ac8-4f0e-9383-babf77982752 None admin] RBAC: Authorizing `identity:create_grant(project_id=ef03f831e4b94427a2be5482f556af22, group_id=e0dfdbe067844c5788d85b1e7040bd19, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:32.726176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea572ce4-0ac8-4f0e-9383-babf77982752 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:32.751316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 213/411] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Sep 6 22:29:32 2022] PUT /identity/v3/projects/ef03f831e4b94427a2be5482f556af22/groups/e0dfdbe067844c5788d85b1e7040bd19/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:32.851993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84904fe7-ce3d-4f1a-9036-e4ab00e883a9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:32.893371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433d46f5-435a-49b5-8a22-ee0c6e6e80d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:32.894312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d46f5-435a-49b5-8a22-ee0c6e6e80d0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:32.894766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d46f5-435a-49b5-8a22-ee0c6e6e80d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:32.895753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433d46f5-435a-49b5-8a22-ee0c6e6e80d0 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:32.896623 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d46f5-435a-49b5-8a22-ee0c6e6e80d0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:32.897123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d46f5-435a-49b5-8a22-ee0c6e6e80d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:32.898696 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433d46f5-435a-49b5-8a22-ee0c6e6e80d0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:32.900709 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:32.911213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 199/412] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:32.918440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea572ce4-0ac8-4f0e-9383-babf77982752 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:33.012851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:33.064088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea572ce4-0ac8-4f0e-9383-babf77982752 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:33.090459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a0b0a-e1e2-4eb4-8fb1-28cbe75132c9 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:33.090755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a0b0a-e1e2-4eb4-8fb1-28cbe75132c9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:33.090993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718a0b0a-e1e2-4eb4-8fb1-28cbe75132c9 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:33.093916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 200/413] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:33.101036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037b6b9-6e14-4a77-a711-64a945ba91a4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:33.101315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037b6b9-6e14-4a77-a711-64a945ba91a4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:33.101585 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a037b6b9-6e14-4a77-a711-64a945ba91a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:33.102491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa51f0b-c42e-4f73-a572-b6daf1ee3c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:33.103130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa51f0b-c42e-4f73-a572-b6daf1ee3c0b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:33.103406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa51f0b-c42e-4f73-a572-b6daf1ee3c0b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:33.103679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa51f0b-c42e-4f73-a572-b6daf1ee3c0b None admin] PATH_INFO: `/v3/groups/a4e50171f099483b9b32cb7aaad16eea` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:33.104805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa51f0b-c42e-4f73-a572-b6daf1ee3c0b None admin] RBAC: Authorizing `identity:get_group(group_id=a4e50171f099483b9b32cb7aaad16eea)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:33.106240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa51f0b-c42e-4f73-a572-b6daf1ee3c0b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:33.108085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 214/414] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:32 2022] GET /identity/v3/groups/a4e50171f099483b9b32cb7aaad16eea => generated 229 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:33.115817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa51f0b-c42e-4f73-a572-b6daf1ee3c0b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:33.128503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a037b6b9-6e14-4a77-a711-64a945ba91a4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:33.130309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a037b6b9-6e14-4a77-a711-64a945ba91a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:33.206510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 201/415] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:33.259352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa51f0b-c42e-4f73-a572-b6daf1ee3c0b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:33.286746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:33.297037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1500a-c1e4-4871-bc9b-47f5ced40b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:33.297674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1500a-c1e4-4871-bc9b-47f5ced40b80 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:33.297949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1500a-c1e4-4871-bc9b-47f5ced40b80 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:33.298220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b1500a-c1e4-4871-bc9b-47f5ced40b80 None admin] PATH_INFO: `/v3/projects/693d7b3554de4d2c9e0e732722fe61d1` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:33.299376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1500a-c1e4-4871-bc9b-47f5ced40b80 None admin] RBAC: Authorizing `identity:get_project(project_id=693d7b3554de4d2c9e0e732722fe61d1)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:33.300902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b1500a-c1e4-4871-bc9b-47f5ced40b80 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:33.304461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 215/416] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:33 2022] GET /identity/v3/projects/693d7b3554de4d2c9e0e732722fe61d1 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:33.314287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037b6b9-6e14-4a77-a711-64a945ba91a4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:33.445156 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a037b6b9-6e14-4a77-a711-64a945ba91a4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:33.472314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:33.482567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729efcc-3871-44e2-8bd3-9067dfb7a774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:33.483223 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729efcc-3871-44e2-8bd3-9067dfb7a774 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:33.483501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729efcc-3871-44e2-8bd3-9067dfb7a774 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:33.483773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729efcc-3871-44e2-8bd3-9067dfb7a774 None admin] PATH_INFO: `/v3/projects/693d7b3554de4d2c9e0e732722fe61d1/groups/a4e50171f099483b9b32cb7aaad16eea/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:33.485725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729efcc-3871-44e2-8bd3-9067dfb7a774 None admin] RBAC: Authorizing `identity:create_grant(project_id=693d7b3554de4d2c9e0e732722fe61d1, group_id=a4e50171f099483b9b32cb7aaad16eea, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:33.487150 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4729efcc-3871-44e2-8bd3-9067dfb7a774 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:33.504260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 202/417] 10.222.0.39 () {58 vars in 1442 bytes} [Tue Sep 6 22:29:33 2022] PUT /identity/v3/projects/693d7b3554de4d2c9e0e732722fe61d1/groups/a4e50171f099483b9b32cb7aaad16eea/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:33.538428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1500a-c1e4-4871-bc9b-47f5ced40b80 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:33.721769 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b1500a-c1e4-4871-bc9b-47f5ced40b80 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:33.772403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1a611-5f2d-4772-ac62-94009bd711dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:33.773287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1a611-5f2d-4772-ac62-94009bd711dc None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:33.773565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1a611-5f2d-4772-ac62-94009bd711dc None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:33.773834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc1a611-5f2d-4772-ac62-94009bd711dc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:33.785258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc1a611-5f2d-4772-ac62-94009bd711dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-dfc1a611-5f2d-4772-ac62-94009bd711dc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:33.793701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:33.794597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:33.795406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 216/418] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:33.800515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729efcc-3871-44e2-8bd3-9067dfb7a774 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:33.908604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4729efcc-3871-44e2-8bd3-9067dfb7a774 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:33.947355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c4988-38fb-4653-bb0b-d4c31789d4a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:33.947978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c4988-38fb-4653-bb0b-d4c31789d4a9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:33.948259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c4988-38fb-4653-bb0b-d4c31789d4a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:33.948531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c4988-38fb-4653-bb0b-d4c31789d4a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:33.949270 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c4988-38fb-4653-bb0b-d4c31789d4a9 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:33.949571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c4988-38fb-4653-bb0b-d4c31789d4a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:33.951063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c4988-38fb-4653-bb0b-d4c31789d4a9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:33.961899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 203/419] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:33.969946 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1a611-5f2d-4772-ac62-94009bd711dc None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:34.041845 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:34.106064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc1a611-5f2d-4772-ac62-94009bd711dc None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:34.156915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee4322-0e6a-4294-8e14-6039c993c4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:34.157579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee4322-0e6a-4294-8e14-6039c993c4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:34.157858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee4322-0e6a-4294-8e14-6039c993c4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:34.158178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ee4322-0e6a-4294-8e14-6039c993c4d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:34.250814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee4322-0e6a-4294-8e14-6039c993c4d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:34.252265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ee4322-0e6a-4294-8e14-6039c993c4d8 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:34.310896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:34.361885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 217/420] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:34.367299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c4988-38fb-4653-bb0b-d4c31789d4a9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:34.433808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:34.490412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c4988-38fb-4653-bb0b-d4c31789d4a9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:34.528276 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1426e24a-8e66-4277-8bd4-8ffb0b382b39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:34.528942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426e24a-8e66-4277-8bd4-8ffb0b382b39 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:34.529221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426e24a-8e66-4277-8bd4-8ffb0b382b39 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:34.529495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1426e24a-8e66-4277-8bd4-8ffb0b382b39 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:34.530640 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426e24a-8e66-4277-8bd4-8ffb0b382b39 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:34.532072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1426e24a-8e66-4277-8bd4-8ffb0b382b39 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:34.534052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 204/421] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:34 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:34.539826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee4322-0e6a-4294-8e14-6039c993c4d8 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:34.599200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35965eb1-924f-4919-b95f-d03889e254c0 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:34.599538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35965eb1-924f-4919-b95f-d03889e254c0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:34.599756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35965eb1-924f-4919-b95f-d03889e254c0 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:34.601442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 205/422] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:34.606252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3573413-653a-46b7-8a82-45459b3ac539 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:34.606546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3573413-653a-46b7-8a82-45459b3ac539 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:34.606818 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3573413-653a-46b7-8a82-45459b3ac539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:34.642845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c3573413-653a-46b7-8a82-45459b3ac539 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:34.644044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c3573413-653a-46b7-8a82-45459b3ac539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:34.648996 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ee4322-0e6a-4294-8e14-6039c993c4d8 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:34.685203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ff688-0904-410a-bd85-fd3100719113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:34.685834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ff688-0904-410a-bd85-fd3100719113 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:34.686104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ff688-0904-410a-bd85-fd3100719113 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:34.686369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ff688-0904-410a-bd85-fd3100719113 None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:34.687473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ff688-0904-410a-bd85-fd3100719113 None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:34.688915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ff688-0904-410a-bd85-fd3100719113 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:34.690873 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 218/423] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:34 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:34.697391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ff688-0904-410a-bd85-fd3100719113 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:34.740729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:34.746544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 206/424] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:34.870284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ff688-0904-410a-bd85-fd3100719113 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:34.925038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be427f91-dba7-4325-819d-9206cba411dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:34.925580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be427f91-dba7-4325-819d-9206cba411dc None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:34.925854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be427f91-dba7-4325-819d-9206cba411dc None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:34.926708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be427f91-dba7-4325-819d-9206cba411dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:34.927714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be427f91-dba7-4325-819d-9206cba411dc None admin] RBAC: Adding query filter params (group.id=e0dfdbe067844c5788d85b1e7040bd19, role.id=aac9ca64da09463ba9b3966c7188d7ee, scope.project.id=a6cc40a4154043ada48161d66116e3c8) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:34.928018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be427f91-dba7-4325-819d-9206cba411dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:34.929471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be427f91-dba7-4325-819d-9206cba411dc None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:34.935854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 219/425] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Sep 6 22:29:34 2022] GET /identity/v3/role_assignments?group.id=e0dfdbe067844c5788d85b1e7040bd19&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 611 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:35.057318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3573413-653a-46b7-8a82-45459b3ac539 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:35.144497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:35.211945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3573413-653a-46b7-8a82-45459b3ac539 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:35.257598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f253848-961e-47b3-89e3-8ecdfb336569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:35.258571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253848-961e-47b3-89e3-8ecdfb336569 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:35.259045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253848-961e-47b3-89e3-8ecdfb336569 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:35.259508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253848-961e-47b3-89e3-8ecdfb336569 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:35.271212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f253848-961e-47b3-89e3-8ecdfb336569 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-6f253848-961e-47b3-89e3-8ecdfb336569 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:35.276593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:35.279687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:35.280471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:35.283040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 207/426] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:29:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:35.289891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be427f91-dba7-4325-819d-9206cba411dc None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:35.405672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:35.424144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be427f91-dba7-4325-819d-9206cba411dc None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:35.464965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df08bf-f883-474b-9b3f-ad335104ae99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:35.465500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df08bf-f883-474b-9b3f-ad335104ae99 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:35.465773 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df08bf-f883-474b-9b3f-ad335104ae99 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:35.466037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df08bf-f883-474b-9b3f-ad335104ae99 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:35.466693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df08bf-f883-474b-9b3f-ad335104ae99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:35.466987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df08bf-f883-474b-9b3f-ad335104ae99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:35.468345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df08bf-f883-474b-9b3f-ad335104ae99 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:35.478463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 220/427] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:35.480418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2ade1-1fa8-4d3d-940e-76eb02856afc None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:35.480698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2ade1-1fa8-4d3d-940e-76eb02856afc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:35.481000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d2ade1-1fa8-4d3d-940e-76eb02856afc None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:35.482422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 208/428] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:29:35.486083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df08bf-f883-474b-9b3f-ad335104ae99 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:35.488474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2189d-6828-41e5-b8d9-9ad72d0c3da0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:35.488474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2189d-6828-41e5-b8d9-9ad72d0c3da0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:35.488474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2189d-6828-41e5-b8d9-9ad72d0c3da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:35.511738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-fdf2189d-6828-41e5-b8d9-9ad72d0c3da0 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:35.512973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fdf2189d-6828-41e5-b8d9-9ad72d0c3da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:35.594010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 209/429] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:35.608872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df08bf-f883-474b-9b3f-ad335104ae99 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:35.648025 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de9a1e-ac62-4110-9877-78e3efdd4c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:35.648636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de9a1e-ac62-4110-9877-78e3efdd4c68 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:35.648943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de9a1e-ac62-4110-9877-78e3efdd4c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:35.649214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de9a1e-ac62-4110-9877-78e3efdd4c68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:35.669544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:35.729203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de9a1e-ac62-4110-9877-78e3efdd4c68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:35.730586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de9a1e-ac62-4110-9877-78e3efdd4c68 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:35.832174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267beede-9cdb-4931-a94e-68f34c7cbcfc None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:35.833078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267beede-9cdb-4931-a94e-68f34c7cbcfc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:35.833273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267beede-9cdb-4931-a94e-68f34c7cbcfc None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:35.834881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 210/430] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:35.839982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efad31e-1b29-44b3-81ca-7406d327c6b2 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:35.840269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efad31e-1b29-44b3-81ca-7406d327c6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:35.840545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efad31e-1b29-44b3-81ca-7406d327c6b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:35.841444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 221/431] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:35.847861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de9a1e-ac62-4110-9877-78e3efdd4c68 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:35.874086 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3efad31e-1b29-44b3-81ca-7406d327c6b2 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:35.875289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3efad31e-1b29-44b3-81ca-7406d327c6b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:35.910888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:35.926890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:35.975713 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 211/432] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:35.976596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efad31e-1b29-44b3-81ca-7406d327c6b2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:35.977250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de9a1e-ac62-4110-9877-78e3efdd4c68 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:36.019055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09259ec-9fd3-47d9-ae80-6346411cdac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:36.020005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09259ec-9fd3-47d9-ae80-6346411cdac2 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.020426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09259ec-9fd3-47d9-ae80-6346411cdac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.020922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09259ec-9fd3-47d9-ae80-6346411cdac2 None admin] PATH_INFO: `/v3/projects/a6cc40a4154043ada48161d66116e3c8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.022270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09259ec-9fd3-47d9-ae80-6346411cdac2 None admin] RBAC: Authorizing `identity:get_project(project_id=a6cc40a4154043ada48161d66116e3c8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:36.023871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09259ec-9fd3-47d9-ae80-6346411cdac2 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:36.030355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 222/433] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:35 2022] GET /identity/v3/projects/a6cc40a4154043ada48161d66116e3c8 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:36.031395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09259ec-9fd3-47d9-ae80-6346411cdac2 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:36.111769 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efad31e-1b29-44b3-81ca-7406d327c6b2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:36.148713 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad66aa-bb2b-4348-b074-20227b9691ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:36.149647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad66aa-bb2b-4348-b074-20227b9691ab None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.150135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad66aa-bb2b-4348-b074-20227b9691ab None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.150583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bad66aa-bb2b-4348-b074-20227b9691ab None admin] PATH_INFO: `/v3/roles/member` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.151028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09259ec-9fd3-47d9-ae80-6346411cdac2 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:36.161945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bad66aa-bb2b-4348-b074-20227b9691ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2bad66aa-bb2b-4348-b074-20227b9691ab None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:36.167391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:36.168302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 06 22:29:36.169220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:36.170506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:36.177069 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 212/434] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:36.181191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad66aa-bb2b-4348-b074-20227b9691ab None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:36.190641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d78422-ae5d-4edd-ad49-d56eec11c7d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:36.191490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d78422-ae5d-4edd-ad49-d56eec11c7d6 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.192352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d78422-ae5d-4edd-ad49-d56eec11c7d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.193343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d78422-ae5d-4edd-ad49-d56eec11c7d6 None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.194666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d78422-ae5d-4edd-ad49-d56eec11c7d6 None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:36.196179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d78422-ae5d-4edd-ad49-d56eec11c7d6 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:36.202950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 223/435] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:36 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:36.205805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d78422-ae5d-4edd-ad49-d56eec11c7d6 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:36.211451 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:36.296384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bad66aa-bb2b-4348-b074-20227b9691ab None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:36.322877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d78422-ae5d-4edd-ad49-d56eec11c7d6 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:36.340545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10791639-192f-4f19-a731-58ab27acf7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:36.341266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791639-192f-4f19-a731-58ab27acf7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.341556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791639-192f-4f19-a731-58ab27acf7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.341829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10791639-192f-4f19-a731-58ab27acf7b8 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.342497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10791639-192f-4f19-a731-58ab27acf7b8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:36.342798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10791639-192f-4f19-a731-58ab27acf7b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:36.344169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10791639-192f-4f19-a731-58ab27acf7b8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:36.354117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 213/436] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:29:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:36.354918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10791639-192f-4f19-a731-58ab27acf7b8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:36.363145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e279d88-8362-4329-9165-14baa75495aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:36.365193 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279d88-8362-4329-9165-14baa75495aa None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.365689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279d88-8362-4329-9165-14baa75495aa None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.366186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279d88-8362-4329-9165-14baa75495aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.367085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e279d88-8362-4329-9165-14baa75495aa None admin] RBAC: Adding query filter params (group.id=e0dfdbe067844c5788d85b1e7040bd19, role.id=952084593c184a78a952e1797a0b6c6b, scope.project.id=a6cc40a4154043ada48161d66116e3c8) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:36.367566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e279d88-8362-4329-9165-14baa75495aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:36.369359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e279d88-8362-4329-9165-14baa75495aa None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:36.376028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 224/437] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Sep 6 22:29:36 2022] GET /identity/v3/role_assignments?group.id=e0dfdbe067844c5788d85b1e7040bd19&scope.project.id=a6cc40a4154043ada48161d66116e3c8&role.id=952084593c184a78a952e1797a0b6c6b => generated 611 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:36.377148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e279d88-8362-4329-9165-14baa75495aa None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:36.447331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:36.460666 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:36.475242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10791639-192f-4f19-a731-58ab27acf7b8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:36.513301 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41927c0a-87fb-4bce-9d6e-c4da649123aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:36.513934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927c0a-87fb-4bce-9d6e-c4da649123aa None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.514210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927c0a-87fb-4bce-9d6e-c4da649123aa None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.514476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41927c0a-87fb-4bce-9d6e-c4da649123aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.521408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e279d88-8362-4329-9165-14baa75495aa None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:36.526535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41927c0a-87fb-4bce-9d6e-c4da649123aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-41927c0a-87fb-4bce-9d6e-c4da649123aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:36.532461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:36.533434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:36.534226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 214/438] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:29:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:36.534871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84523d-d77f-430b-ad29-88f8ae75d158 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.535494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84523d-d77f-430b-ad29-88f8ae75d158 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.535494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d84523d-d77f-430b-ad29-88f8ae75d158 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.537158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 215/439] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:36.541722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84523d-d77f-430b-ad29-88f8ae75d158 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:36.569743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94832-b270-46f2-8a1b-8df895348d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:36.570280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94832-b270-46f2-8a1b-8df895348d1c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.570564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94832-b270-46f2-8a1b-8df895348d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.570850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba94832-b270-46f2-8a1b-8df895348d1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.665486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d84523d-d77f-430b-ad29-88f8ae75d158 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:36.666569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba94832-b270-46f2-8a1b-8df895348d1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:36.667999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba94832-b270-46f2-8a1b-8df895348d1c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:36.712689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971f802-e057-4ace-b91a-0a746452ee24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:36.713733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971f802-e057-4ace-b91a-0a746452ee24 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.714013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971f802-e057-4ace-b91a-0a746452ee24 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.714288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971f802-e057-4ace-b91a-0a746452ee24 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.714950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971f802-e057-4ace-b91a-0a746452ee24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:36.715419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971f802-e057-4ace-b91a-0a746452ee24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:36.716609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971f802-e057-4ace-b91a-0a746452ee24 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:36.721738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:36.726706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 216/440] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:36.727943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27e6c7-b858-42b1-9207-d17913f89739 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.728190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27e6c7-b858-42b1-9207-d17913f89739 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.728458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f27e6c7-b858-42b1-9207-d17913f89739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.750316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1f27e6c7-b858-42b1-9207-d17913f89739 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:36.751533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1f27e6c7-b858-42b1-9207-d17913f89739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:36.755228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:36.783930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 225/441] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:36.785030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94832-b270-46f2-8a1b-8df895348d1c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:36.828817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 217/442] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:36.829601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27e6c7-b858-42b1-9207-d17913f89739 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:36.891132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba94832-b270-46f2-8a1b-8df895348d1c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:36.925816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda17a45-c2ac-464d-90c1-39008ae64e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:36.927182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17a45-c2ac-464d-90c1-39008ae64e6b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.927462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17a45-c2ac-464d-90c1-39008ae64e6b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.927740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda17a45-c2ac-464d-90c1-39008ae64e6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.942795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f27e6c7-b858-42b1-9207-d17913f89739 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:36.945332 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:36.977748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720b2d5-6ee2-4563-a864-55fcc6a13aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:36.978481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720b2d5-6ee2-4563-a864-55fcc6a13aa7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:36.978759 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720b2d5-6ee2-4563-a864-55fcc6a13aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:36.979027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1720b2d5-6ee2-4563-a864-55fcc6a13aa7 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:36.980140 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720b2d5-6ee2-4563-a864-55fcc6a13aa7 None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:36.981593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1720b2d5-6ee2-4563-a864-55fcc6a13aa7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:36.987647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 218/443] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:36 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:36.988667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720b2d5-6ee2-4563-a864-55fcc6a13aa7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:36.999963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:37.001174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda17a45-c2ac-464d-90c1-39008ae64e6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:37.002617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda17a45-c2ac-464d-90c1-39008ae64e6b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:37.094672 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1720b2d5-6ee2-4563-a864-55fcc6a13aa7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:37.107558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 226/444] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:37.108386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda17a45-c2ac-464d-90c1-39008ae64e6b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:37.131428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38035668-027d-473f-b0d2-e4cce7c69a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:37.132058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38035668-027d-473f-b0d2-e4cce7c69a92 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:37.132598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38035668-027d-473f-b0d2-e4cce7c69a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:37.132907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38035668-027d-473f-b0d2-e4cce7c69a92 None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:37.134031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38035668-027d-473f-b0d2-e4cce7c69a92 None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:37.135468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38035668-027d-473f-b0d2-e4cce7c69a92 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:37.142083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 219/445] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:36 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:37.142083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38035668-027d-473f-b0d2-e4cce7c69a92 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:37.158467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:37.208235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda17a45-c2ac-464d-90c1-39008ae64e6b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:37.212692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:37.235092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38035668-027d-473f-b0d2-e4cce7c69a92 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:37.242031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba1b09-4727-4127-a18f-ad791a1325b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:37.242659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba1b09-4727-4127-a18f-ad791a1325b4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:37.242928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba1b09-4727-4127-a18f-ad791a1325b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:37.243199 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ba1b09-4727-4127-a18f-ad791a1325b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:37.253610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ba1b09-4727-4127-a18f-ad791a1325b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-55ba1b09-4727-4127-a18f-ad791a1325b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:37.258801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:37.259729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:37.260522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 227/446] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:29:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:37.261572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba1b09-4727-4127-a18f-ad791a1325b4 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:37.269394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963337d-e341-4371-8797-4e8d1bd85d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:37.270536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963337d-e341-4371-8797-4e8d1bd85d86 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:37.270810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963337d-e341-4371-8797-4e8d1bd85d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:37.271078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963337d-e341-4371-8797-4e8d1bd85d86 None admin] PATH_INFO: `/v3/projects/ef03f831e4b94427a2be5482f556af22` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:37.272197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963337d-e341-4371-8797-4e8d1bd85d86 None admin] RBAC: Authorizing `identity:get_project(project_id=ef03f831e4b94427a2be5482f556af22)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:37.273701 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4963337d-e341-4371-8797-4e8d1bd85d86 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:37.275309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 220/447] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:37 2022] GET /identity/v3/projects/ef03f831e4b94427a2be5482f556af22 => generated 302 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:37.276047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963337d-e341-4371-8797-4e8d1bd85d86 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:37.364005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ba1b09-4727-4127-a18f-ad791a1325b4 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:37.372828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:37.375517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963337d-e341-4371-8797-4e8d1bd85d86 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:37.398944 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596af61-eac9-4d63-b720-f79dac3bba69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:37.399637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596af61-eac9-4d63-b720-f79dac3bba69 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:37.399999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596af61-eac9-4d63-b720-f79dac3bba69 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:37.400277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596af61-eac9-4d63-b720-f79dac3bba69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:37.401023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596af61-eac9-4d63-b720-f79dac3bba69 None admin] RBAC: Adding query filter params (group.id=e0dfdbe067844c5788d85b1e7040bd19, role.id=aac9ca64da09463ba9b3966c7188d7ee, scope.project.id=ef03f831e4b94427a2be5482f556af22) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:37.401330 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596af61-eac9-4d63-b720-f79dac3bba69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:37.402758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d596af61-eac9-4d63-b720-f79dac3bba69 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:37.408279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 228/448] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Sep 6 22:29:37 2022] GET /identity/v3/role_assignments?group.id=e0dfdbe067844c5788d85b1e7040bd19&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=aac9ca64da09463ba9b3966c7188d7ee => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:37.409139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596af61-eac9-4d63-b720-f79dac3bba69 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:37.409305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df5cac-20ca-41bc-937d-e6ad81e22fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:37.409898 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df5cac-20ca-41bc-937d-e6ad81e22fa9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:37.410128 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df5cac-20ca-41bc-937d-e6ad81e22fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:37.411197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69df5cac-20ca-41bc-937d-e6ad81e22fa9 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:37.412208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df5cac-20ca-41bc-937d-e6ad81e22fa9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:37.412516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df5cac-20ca-41bc-937d-e6ad81e22fa9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:37.413954 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69df5cac-20ca-41bc-937d-e6ad81e22fa9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:37.424170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 221/449] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:37.430677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df5cac-20ca-41bc-937d-e6ad81e22fa9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:37.443448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:37.532694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596af61-eac9-4d63-b720-f79dac3bba69 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:37.548626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69df5cac-20ca-41bc-937d-e6ad81e22fa9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:37.580649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45824f5e-ecd1-41e5-9015-e1dbc9fd4130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:37.581657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45824f5e-ecd1-41e5-9015-e1dbc9fd4130 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:37.581940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45824f5e-ecd1-41e5-9015-e1dbc9fd4130 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:37.582272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45824f5e-ecd1-41e5-9015-e1dbc9fd4130 None admin] PATH_INFO: `/v3/groups/e0dfdbe067844c5788d85b1e7040bd19` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:37.583472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45824f5e-ecd1-41e5-9015-e1dbc9fd4130 None admin] RBAC: Authorizing `identity:get_group(group_id=e0dfdbe067844c5788d85b1e7040bd19)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:37.584955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45824f5e-ecd1-41e5-9015-e1dbc9fd4130 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:37.586604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 229/450] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:37 2022] GET /identity/v3/groups/e0dfdbe067844c5788d85b1e7040bd19 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:37.587524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9afb8-5534-48be-8cd1-f6f32afbcd72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:37.588202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9afb8-5534-48be-8cd1-f6f32afbcd72 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:37.588479 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9afb8-5534-48be-8cd1-f6f32afbcd72 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:37.588745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f9afb8-5534-48be-8cd1-f6f32afbcd72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:37.592789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45824f5e-ecd1-41e5-9015-e1dbc9fd4130 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:37.639794 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:37.671905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9afb8-5534-48be-8cd1-f6f32afbcd72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:37.673399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f9afb8-5534-48be-8cd1-f6f32afbcd72 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:37.681601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:37.711860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45824f5e-ecd1-41e5-9015-e1dbc9fd4130 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:37.755667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a700a-0433-465a-bd0c-400dcc54093a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:37.756566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a700a-0433-465a-bd0c-400dcc54093a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:37.757040 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a700a-0433-465a-bd0c-400dcc54093a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:37.757704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a700a-0433-465a-bd0c-400dcc54093a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:37.758842 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a700a-0433-465a-bd0c-400dcc54093a None admin] RBAC: Adding query filter params (group.id=e0dfdbe067844c5788d85b1e7040bd19, role.id=952084593c184a78a952e1797a0b6c6b, scope.project.id=ef03f831e4b94427a2be5482f556af22) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:37.759313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a700a-0433-465a-bd0c-400dcc54093a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:37.761711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a700a-0433-465a-bd0c-400dcc54093a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:37.770678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 230/451] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Sep 6 22:29:37 2022] GET /identity/v3/role_assignments?group.id=e0dfdbe067844c5788d85b1e7040bd19&scope.project.id=ef03f831e4b94427a2be5482f556af22&role.id=952084593c184a78a952e1797a0b6c6b => generated 611 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:37.800034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 222/452] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 22:29:37.805889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a700a-0433-465a-bd0c-400dcc54093a None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:37.916966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a700a-0433-465a-bd0c-400dcc54093a None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:37.919386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:37.954548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3be41-9b06-4e00-8e39-2e17e0193954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:37.955177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3be41-9b06-4e00-8e39-2e17e0193954 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:37.955454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3be41-9b06-4e00-8e39-2e17e0193954 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:37.959218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3be41-9b06-4e00-8e39-2e17e0193954 None admin] PATH_INFO: `/v3/projects/693d7b3554de4d2c9e0e732722fe61d1` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:37.959218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3be41-9b06-4e00-8e39-2e17e0193954 None admin] RBAC: Authorizing `identity:get_project(project_id=693d7b3554de4d2c9e0e732722fe61d1)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:37.959218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca3be41-9b06-4e00-8e39-2e17e0193954 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:37.960637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 231/453] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:29:37 2022] GET /identity/v3/projects/693d7b3554de4d2c9e0e732722fe61d1 => generated 344 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:37.966207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9afb8-5534-48be-8cd1-f6f32afbcd72 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:38.065596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f9afb8-5534-48be-8cd1-f6f32afbcd72 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:38.074940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:38.106194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831b7ea1-d5e6-48ad-a11b-40cd97457538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:38.106864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7ea1-d5e6-48ad-a11b-40cd97457538 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:38.107167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7ea1-d5e6-48ad-a11b-40cd97457538 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:38.107440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831b7ea1-d5e6-48ad-a11b-40cd97457538 None admin] PATH_INFO: `/v3/groups/a4e50171f099483b9b32cb7aaad16eea` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:38.108703 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b7ea1-d5e6-48ad-a11b-40cd97457538 None admin] RBAC: Authorizing `identity:get_group(group_id=a4e50171f099483b9b32cb7aaad16eea)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:38.110190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831b7ea1-d5e6-48ad-a11b-40cd97457538 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:38.111753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 223/454] 10.222.0.39 () {56 vars in 1180 bytes} [Tue Sep 6 22:29:37 2022] GET /identity/v3/groups/a4e50171f099483b9b32cb7aaad16eea => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:38.117710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3be41-9b06-4e00-8e39-2e17e0193954 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:38.218601 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3be41-9b06-4e00-8e39-2e17e0193954 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:38.250436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500c42fa-c106-43c9-98c1-618d4de09843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:38.251030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c42fa-c106-43c9-98c1-618d4de09843 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:38.251308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c42fa-c106-43c9-98c1-618d4de09843 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:38.251579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500c42fa-c106-43c9-98c1-618d4de09843 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:38.252292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c42fa-c106-43c9-98c1-618d4de09843 None admin] RBAC: Adding query filter params (group.id=a4e50171f099483b9b32cb7aaad16eea, role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=693d7b3554de4d2c9e0e732722fe61d1) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:38.252598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c42fa-c106-43c9-98c1-618d4de09843 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:38.254050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500c42fa-c106-43c9-98c1-618d4de09843 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:38.260595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 232/455] 10.222.0.39 () {56 vars in 1376 bytes} [Tue Sep 6 22:29:38 2022] GET /identity/v3/role_assignments?group.id=a4e50171f099483b9b32cb7aaad16eea&scope.project.id=693d7b3554de4d2c9e0e732722fe61d1&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:40.790260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde24689-2c61-4532-848d-83c7197d8a2f None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:40.790774 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde24689-2c61-4532-848d-83c7197d8a2f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:40.791056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde24689-2c61-4532-848d-83c7197d8a2f None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:40.792617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 224/456] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:40.798109 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e852cc-4b6a-4c0c-9605-c2e4633d86b8 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:40.798913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e852cc-4b6a-4c0c-9605-c2e4633d86b8 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:40.799423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e852cc-4b6a-4c0c-9605-c2e4633d86b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:40.819287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-65e852cc-4b6a-4c0c-9605-c2e4633d86b8 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:40.820853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-65e852cc-4b6a-4c0c-9605-c2e4633d86b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:40.867267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:40.917037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 233/457] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:41.121248 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb9938-64c2-46cd-8015-5dcfeb400bec None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:41.121894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb9938-64c2-46cd-8015-5dcfeb400bec None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:41.122385 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84eb9938-64c2-46cd-8015-5dcfeb400bec None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:41.124129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 225/458] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:41.132791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf8ef-7e96-410e-a303-95d90b2c7cd0 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:41.132791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf8ef-7e96-410e-a303-95d90b2c7cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:41.132791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37bf8ef-7e96-410e-a303-95d90b2c7cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:41.153687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-a37bf8ef-7e96-410e-a303-95d90b2c7cd0 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:41.156114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a37bf8ef-7e96-410e-a303-95d90b2c7cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:41.250589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 234/459] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:41.337454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6405f0-95fb-4531-9500-0ce432de0fe9 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:41.337796 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6405f0-95fb-4531-9500-0ce432de0fe9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:41.338005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6405f0-95fb-4531-9500-0ce432de0fe9 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:41.339590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 226/460] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:41.344995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c9d35-de90-423e-af73-94bffe58d4fe None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:41.345280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c9d35-de90-423e-af73-94bffe58d4fe None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:41.345543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6405f0-95fb-4531-9500-0ce432de0fe9 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:41.345658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c9d35-de90-423e-af73-94bffe58d4fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:41.369655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-458c9d35-de90-423e-af73-94bffe58d4fe None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:41.371899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-458c9d35-de90-423e-af73-94bffe58d4fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:41.501862 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6405f0-95fb-4531-9500-0ce432de0fe9 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:41.523901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 235/461] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:41.554688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166b06e-5214-462b-9d88-c45b5607c08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:41.555374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166b06e-5214-462b-9d88-c45b5607c08d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:41.555660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166b06e-5214-462b-9d88-c45b5607c08d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:41.555934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9166b06e-5214-462b-9d88-c45b5607c08d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:41.568478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166b06e-5214-462b-9d88-c45b5607c08d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:41.570017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9166b06e-5214-462b-9d88-c45b5607c08d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-9166b06e-5214-462b-9d88-c45b5607c08d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:41.598028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:41.598941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:41.600528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:41.600528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:41.600528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:41.600528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 227/462] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:41.600743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c9d35-de90-423e-af73-94bffe58d4fe None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:41.605873 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166b06e-5214-462b-9d88-c45b5607c08d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:41.651495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:41.672197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:41.759639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9166b06e-5214-462b-9d88-c45b5607c08d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:41.812671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95d286-925e-4ee6-932c-f521f8bf9dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:41.813459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95d286-925e-4ee6-932c-f521f8bf9dec None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:41.813643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95d286-925e-4ee6-932c-f521f8bf9dec None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:41.813920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95d286-925e-4ee6-932c-f521f8bf9dec None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:41.814567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95d286-925e-4ee6-932c-f521f8bf9dec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:41.814867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95d286-925e-4ee6-932c-f521f8bf9dec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:41.816239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95d286-925e-4ee6-932c-f521f8bf9dec None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:41.825647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c9d35-de90-423e-af73-94bffe58d4fe None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:41.830294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 228/463] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:41.835407 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95d286-925e-4ee6-932c-f521f8bf9dec None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:41.896359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5f368-8202-4380-a688-4c5e5db25d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:41.897033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5f368-8202-4380-a688-4c5e5db25d33 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:41.897475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5f368-8202-4380-a688-4c5e5db25d33 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:41.898003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad5f368-8202-4380-a688-4c5e5db25d33 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:41.911268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5f368-8202-4380-a688-4c5e5db25d33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:41.913768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad5f368-8202-4380-a688-4c5e5db25d33 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-dad5f368-8202-4380-a688-4c5e5db25d33 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:41.931951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:41.932976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:41.933795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:41.933795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:41.933795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:41.940477 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 236/464] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:41.941397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5f368-8202-4380-a688-4c5e5db25d33 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:41.970834 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:41.992284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95d286-925e-4ee6-932c-f521f8bf9dec None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:42.029534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e906cf22-4506-4037-a069-57751bff3a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:42.030145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906cf22-4506-4037-a069-57751bff3a9f None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:42.030420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906cf22-4506-4037-a069-57751bff3a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:42.030684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e906cf22-4506-4037-a069-57751bff3a9f None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:42.031358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906cf22-4506-4037-a069-57751bff3a9f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:42.033044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e906cf22-4506-4037-a069-57751bff3a9f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:42.034742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:42.057149 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 229/465] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:29:41 2022] POST /identity/v3/users => generated 246 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:29:42.067949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad5f368-8202-4380-a688-4c5e5db25d33 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:42.070715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6e480-619b-4207-8c98-e8a5a6bfb59a None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:42.070999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6e480-619b-4207-8c98-e8a5a6bfb59a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:42.071271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac6e480-619b-4207-8c98-e8a5a6bfb59a None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:42.077125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 230/466] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:42 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:42.077125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6e480-619b-4207-8c98-e8a5a6bfb59a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:42.117624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09286fe3-300a-4af7-95b5-e0f0407f527a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:42.118285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09286fe3-300a-4af7-95b5-e0f0407f527a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:42.118561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09286fe3-300a-4af7-95b5-e0f0407f527a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:42.118824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09286fe3-300a-4af7-95b5-e0f0407f527a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:42.135300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09286fe3-300a-4af7-95b5-e0f0407f527a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:42.136789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09286fe3-300a-4af7-95b5-e0f0407f527a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-09286fe3-300a-4af7-95b5-e0f0407f527a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:42.158162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:42.159162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:42.160000 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:42.160000 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:42.160000 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:42.167380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 237/467] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:42.168697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ef418-354d-4f6a-a057-91a35b679782 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:42.169006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ef418-354d-4f6a-a057-91a35b679782 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:42.169279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-888ef418-354d-4f6a-a057-91a35b679782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:42.195956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-888ef418-354d-4f6a-a057-91a35b679782 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:42.197793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-888ef418-354d-4f6a-a057-91a35b679782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:42.235064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac6e480-619b-4207-8c98-e8a5a6bfb59a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:42.287071 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c7007-817c-494c-bc4e-12ea97d04de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:42.287678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c7007-817c-494c-bc4e-12ea97d04de7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:42.287957 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c7007-817c-494c-bc4e-12ea97d04de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:42.288224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2c7007-817c-494c-bc4e-12ea97d04de7 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:42.288904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c7007-817c-494c-bc4e-12ea97d04de7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:42.289202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c7007-817c-494c-bc4e-12ea97d04de7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:42.290578 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2c7007-817c-494c-bc4e-12ea97d04de7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:42.300074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 238/468] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:42.302471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ef418-354d-4f6a-a057-91a35b679782 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:42.315509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 231/469] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:42.317248 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c7007-817c-494c-bc4e-12ea97d04de7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:42.325863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:42.407995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:42.462224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-888ef418-354d-4f6a-a057-91a35b679782 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:42.479087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2c7007-817c-494c-bc4e-12ea97d04de7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:42.508202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b291058-73c4-46e1-8633-32cfe126f199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:42.508858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b291058-73c4-46e1-8633-32cfe126f199 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:42.509135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b291058-73c4-46e1-8633-32cfe126f199 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:42.509402 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b291058-73c4-46e1-8633-32cfe126f199 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:42.510053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b291058-73c4-46e1-8633-32cfe126f199 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:42.510352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b291058-73c4-46e1-8633-32cfe126f199 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:42.512026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b291058-73c4-46e1-8633-32cfe126f199 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:42.527512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 239/470] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:42.529012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69a636-67da-4370-a22a-a0bacb3d79b8 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:42.529295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69a636-67da-4370-a22a-a0bacb3d79b8 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:42.529569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69a636-67da-4370-a22a-a0bacb3d79b8 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:42.530604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea2abd0-3904-41b3-a6bf-7f8c829d6633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:42.531068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 240/471] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:42.531721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2abd0-3904-41b3-a6bf-7f8c829d6633 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:42.532250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2abd0-3904-41b3-a6bf-7f8c829d6633 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:42.533467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69a636-67da-4370-a22a-a0bacb3d79b8 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:42.533718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2abd0-3904-41b3-a6bf-7f8c829d6633 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:42.534574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2abd0-3904-41b3-a6bf-7f8c829d6633 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:42.537109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea2abd0-3904-41b3-a6bf-7f8c829d6633 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:42.581092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 232/472] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:29:42 2022] POST /identity/v3/users => generated 248 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:29:42.587387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4bdc0-ae95-422f-9c3d-c49f5a194440 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:42.588227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4bdc0-ae95-422f-9c3d-c49f5a194440 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:42.589179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca4bdc0-ae95-422f-9c3d-c49f5a194440 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:42.628362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-bca4bdc0-ae95-422f-9c3d-c49f5a194440 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:42.630650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bca4bdc0-ae95-422f-9c3d-c49f5a194440 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:42.655761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69a636-67da-4370-a22a-a0bacb3d79b8 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:42.686312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:42.697331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:42.707005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bae8e-f4bc-4307-8ee5-828cd1a018bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:42.707886 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bae8e-f4bc-4307-8ee5-828cd1a018bd None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:42.708159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bae8e-f4bc-4307-8ee5-828cd1a018bd None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:42.708426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581bae8e-f4bc-4307-8ee5-828cd1a018bd None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:42.709801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bae8e-f4bc-4307-8ee5-828cd1a018bd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:42.711573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581bae8e-f4bc-4307-8ee5-828cd1a018bd None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:42.718239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 233/473] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:42.719531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4bdc0-ae95-422f-9c3d-c49f5a194440 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:42.760795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 241/474] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:29:42 2022] POST /identity/v3/users => generated 248 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:29:42.880247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca4bdc0-ae95-422f-9c3d-c49f5a194440 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:42.922370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85af658-996f-40d4-9937-e0e3c39631fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:42.923112 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85af658-996f-40d4-9937-e0e3c39631fd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:42.923363 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85af658-996f-40d4-9937-e0e3c39631fd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:42.923632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85af658-996f-40d4-9937-e0e3c39631fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:42.936447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85af658-996f-40d4-9937-e0e3c39631fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:42.937949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85af658-996f-40d4-9937-e0e3c39631fd None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a85af658-996f-40d4-9937-e0e3c39631fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:42.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:42.955152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:42.955973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:42.955973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:42.955973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:42.955973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 234/475] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:42.962918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bae8e-f4bc-4307-8ee5-828cd1a018bd None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:43.077440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85af658-996f-40d4-9937-e0e3c39631fd None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:43.081752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581bae8e-f4bc-4307-8ee5-828cd1a018bd None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:43.125404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789d50c-aba6-4178-8a5b-ec9a1f6d0534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:43.126013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789d50c-aba6-4178-8a5b-ec9a1f6d0534 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:43.126287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789d50c-aba6-4178-8a5b-ec9a1f6d0534 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:43.126563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0789d50c-aba6-4178-8a5b-ec9a1f6d0534 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:43.127663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789d50c-aba6-4178-8a5b-ec9a1f6d0534 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:43.127960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789d50c-aba6-4178-8a5b-ec9a1f6d0534 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:43.129847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0789d50c-aba6-4178-8a5b-ec9a1f6d0534 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:43.134697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:43.147343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 242/476] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:43.153052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789d50c-aba6-4178-8a5b-ec9a1f6d0534 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:43.190381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:43.220041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85af658-996f-40d4-9937-e0e3c39631fd None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:43.294230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eded67-f600-4099-8d3e-bfa8aba59660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:43.295982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eded67-f600-4099-8d3e-bfa8aba59660 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:43.298201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eded67-f600-4099-8d3e-bfa8aba59660 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:43.298877 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01eded67-f600-4099-8d3e-bfa8aba59660 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:43.302830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eded67-f600-4099-8d3e-bfa8aba59660 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:43.305879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01eded67-f600-4099-8d3e-bfa8aba59660 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:43.319780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.sql.core [None req-01eded67-f600-4099-8d3e-bfa8aba59660 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 22:29:43.319780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 22:29:43.319780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [parameters: {'id': '1f342a88e20f4c11bf614830346a7d51', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 22:29:43.319780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81665) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 22:29:43.334284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0789d50c-aba6-4178-8a5b-ec9a1f6d0534 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-01eded67-f600-4099-8d3e-bfa8aba59660 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application conn.query(q) Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result.read() Sep 06 22:29:43.341367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return ref.to_dict() Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 22:29:43.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application next(self.gen) Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application yield resource Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application next(self.gen) Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise self.value Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application session.commit() Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 22:29:43.343627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.session.flush() Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application compat.raise_( Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 22:29:43.344670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 22:29:43.345756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application conn.query(q) Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result.read() Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 22:29:43.346745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [parameters: {'id': '1f342a88e20f4c11bf614830346a7d51', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:43.347697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 06 22:29:43.353712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:43.354699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 235/477] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Sep 6 22:29:43 2022] POST /identity/v3/roles => generated 144 bytes in 272 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:43.358562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eded67-f600-4099-8d3e-bfa8aba59660 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:43.425826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90163370-34ba-429c-a103-49e346b0b861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:43.426490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90163370-34ba-429c-a103-49e346b0b861 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:43.426768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90163370-34ba-429c-a103-49e346b0b861 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:43.427043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90163370-34ba-429c-a103-49e346b0b861 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:43.427689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90163370-34ba-429c-a103-49e346b0b861 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:43.432984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90163370-34ba-429c-a103-49e346b0b861 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:43.477436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 243/478] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:29:43 2022] POST /identity/v3/users => generated 249 bytes in 325 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:29:43.542878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01eded67-f600-4099-8d3e-bfa8aba59660 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:43.555375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:43.622241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce982e2a-556d-4a98-b480-8f9b8e53e760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:43.622958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce982e2a-556d-4a98-b480-8f9b8e53e760 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:43.623241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce982e2a-556d-4a98-b480-8f9b8e53e760 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:43.623565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce982e2a-556d-4a98-b480-8f9b8e53e760 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:43.639771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce982e2a-556d-4a98-b480-8f9b8e53e760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-ce982e2a-556d-4a98-b480-8f9b8e53e760 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:43.644878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:43.645812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:43.646593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 236/479] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:29:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:43.651972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90163370-34ba-429c-a103-49e346b0b861 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:43.762277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:43.772452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90163370-34ba-429c-a103-49e346b0b861 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:43.810760 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562d87e8-5bfa-48ad-8cf0-e057410fea66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:43.811728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d87e8-5bfa-48ad-8cf0-e057410fea66 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:43.812196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d87e8-5bfa-48ad-8cf0-e057410fea66 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:43.812686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562d87e8-5bfa-48ad-8cf0-e057410fea66 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:43.813570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d87e8-5bfa-48ad-8cf0-e057410fea66 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:43.814085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d87e8-5bfa-48ad-8cf0-e057410fea66 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:43.815634 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562d87e8-5bfa-48ad-8cf0-e057410fea66 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:43.824952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 244/480] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:44.813803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705eeb-25be-41bf-a994-403c76cbf998 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:44.814306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705eeb-25be-41bf-a994-403c76cbf998 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:44.814405 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64705eeb-25be-41bf-a994-403c76cbf998 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:44.815923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 237/481] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:44.821572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4f967-7635-4964-a6e1-862603ae190c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:44.822037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4f967-7635-4964-a6e1-862603ae190c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:44.822227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d4f967-7635-4964-a6e1-862603ae190c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:44.842217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-b5d4f967-7635-4964-a6e1-862603ae190c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:44.843333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b5d4f967-7635-4964-a6e1-862603ae190c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:44.937473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 245/482] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:45.243224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64705eeb-25be-41bf-a994-403c76cbf998 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:45.389644 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64705eeb-25be-41bf-a994-403c76cbf998 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:45.437733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5656c-f5ff-4368-a353-bf3d38c2e0cb None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:45.438048 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5656c-f5ff-4368-a353-bf3d38c2e0cb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:45.438274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5656c-f5ff-4368-a353-bf3d38c2e0cb None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:45.439322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b75d6-10d8-4648-9e79-29ace724bbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:45.439955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b75d6-10d8-4648-9e79-29ace724bbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:45.440231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b75d6-10d8-4648-9e79-29ace724bbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:45.440499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65b75d6-10d8-4648-9e79-29ace724bbd2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:45.441106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 246/483] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:45.446038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48c1f8-d7ca-4b64-8dba-cad188f82ece None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:45.446316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48c1f8-d7ca-4b64-8dba-cad188f82ece None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:45.446586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c48c1f8-d7ca-4b64-8dba-cad188f82ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:45.457094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65b75d6-10d8-4648-9e79-29ace724bbd2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-b65b75d6-10d8-4648-9e79-29ace724bbd2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:45.462201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:45.463072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:45.463942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 238/484] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:45.469981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-8c48c1f8-d7ca-4b64-8dba-cad188f82ece None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:45.470498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b75d6-10d8-4648-9e79-29ace724bbd2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:45.471172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8c48c1f8-d7ca-4b64-8dba-cad188f82ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:45.505102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:45.561697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 247/485] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:45.563343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a7335-0c60-4837-8d95-a8ae14f98478 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:45.563817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a7335-0c60-4837-8d95-a8ae14f98478 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:45.564280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919a7335-0c60-4837-8d95-a8ae14f98478 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:45.566092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 248/486] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:45.574847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed762f22-8bde-4ecd-92fd-74ba2e2b95d6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:45.575955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed762f22-8bde-4ecd-92fd-74ba2e2b95d6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:45.575955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed762f22-8bde-4ecd-92fd-74ba2e2b95d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:45.596062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-ed762f22-8bde-4ecd-92fd-74ba2e2b95d6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:45.597312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ed762f22-8bde-4ecd-92fd-74ba2e2b95d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:45.609430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:45.648227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65b75d6-10d8-4648-9e79-29ace724bbd2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:45.678355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 249/487] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:45.701595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc839d2-165c-49cb-8ea6-bd1c8b7baffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:45.702660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc839d2-165c-49cb-8ea6-bd1c8b7baffa None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:45.703113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc839d2-165c-49cb-8ea6-bd1c8b7baffa None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:45.703556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc839d2-165c-49cb-8ea6-bd1c8b7baffa None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:45.704683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc839d2-165c-49cb-8ea6-bd1c8b7baffa None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:45.705063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc839d2-165c-49cb-8ea6-bd1c8b7baffa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:45.706440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc839d2-165c-49cb-8ea6-bd1c8b7baffa None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:45.716558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 239/488] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:29:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:45.722093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed762f22-8bde-4ecd-92fd-74ba2e2b95d6 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:45.826100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed762f22-8bde-4ecd-92fd-74ba2e2b95d6 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:45.860313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9312f-f2a2-47fb-a5b2-b8ef3c3e3080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:45.861025 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9312f-f2a2-47fb-a5b2-b8ef3c3e3080 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:45.861295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9312f-f2a2-47fb-a5b2-b8ef3c3e3080 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:45.861560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9312f-f2a2-47fb-a5b2-b8ef3c3e3080 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:45.872266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9312f-f2a2-47fb-a5b2-b8ef3c3e3080 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:45.873726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb9312f-f2a2-47fb-a5b2-b8ef3c3e3080 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:45.876252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-5cb9312f-f2a2-47fb-a5b2-b8ef3c3e3080 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:45.889784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:45.890687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:45.891547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:45.891547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:45.891547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:45.891547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 250/489] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:45.896653 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc839d2-165c-49cb-8ea6-bd1c8b7baffa None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:45.898473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9312f-f2a2-47fb-a5b2-b8ef3c3e3080 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:45.997488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:46.017069 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc839d2-165c-49cb-8ea6-bd1c8b7baffa None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:46.055889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63e3b9-55fd-4295-9862-da57de320d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:46.057314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63e3b9-55fd-4295-9862-da57de320d02 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.057590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63e3b9-55fd-4295-9862-da57de320d02 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.057860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63e3b9-55fd-4295-9862-da57de320d02 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.058503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63e3b9-55fd-4295-9862-da57de320d02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:46.058799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63e3b9-55fd-4295-9862-da57de320d02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:46.060165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63e3b9-55fd-4295-9862-da57de320d02 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:46.074082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 240/490] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:46.079581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63e3b9-55fd-4295-9862-da57de320d02 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:46.141027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9312f-f2a2-47fb-a5b2-b8ef3c3e3080 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:46.194686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62775f3f-9c2a-45d0-8fed-3e7f45e9c141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:46.195147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63e3b9-55fd-4295-9862-da57de320d02 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:46.195331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775f3f-9c2a-45d0-8fed-3e7f45e9c141 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.195598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775f3f-9c2a-45d0-8fed-3e7f45e9c141 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.195873 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62775f3f-9c2a-45d0-8fed-3e7f45e9c141 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.209920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62775f3f-9c2a-45d0-8fed-3e7f45e9c141 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-62775f3f-9c2a-45d0-8fed-3e7f45e9c141 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:46.216725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:46.217638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:46.218426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:46.220907 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 251/491] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:46.221720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62775f3f-9c2a-45d0-8fed-3e7f45e9c141 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:46.238136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea47011-b8ee-41fa-a331-6b89a111e629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:46.240047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea47011-b8ee-41fa-a331-6b89a111e629 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.240341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea47011-b8ee-41fa-a331-6b89a111e629 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.240615 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea47011-b8ee-41fa-a331-6b89a111e629 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.248245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea47011-b8ee-41fa-a331-6b89a111e629 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:46.249738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea47011-b8ee-41fa-a331-6b89a111e629 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-dea47011-b8ee-41fa-a331-6b89a111e629 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:29:46.264490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:46.265447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:46.266328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:46.266328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:46.266328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:29:46.266328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:29:46.266328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:29:46.266328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:29:46.266328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:29:46.266328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:29:46.266328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:29:46.266328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:46.271541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 241/492] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Sep 6 22:29:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:46.272416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea47011-b8ee-41fa-a331-6b89a111e629 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:46.282249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:46.293267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:46.340005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62775f3f-9c2a-45d0-8fed-3e7f45e9c141 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:46.379919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1326e8-6be8-47c3-8365-48ba528d4dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:46.381152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1326e8-6be8-47c3-8365-48ba528d4dab None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.381629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1326e8-6be8-47c3-8365-48ba528d4dab None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.382127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1326e8-6be8-47c3-8365-48ba528d4dab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.388136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea47011-b8ee-41fa-a331-6b89a111e629 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:46.396705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1326e8-6be8-47c3-8365-48ba528d4dab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-7e1326e8-6be8-47c3-8365-48ba528d4dab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:46.402337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:46.403283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:46.404145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:46.406624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 252/493] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:46.407607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1326e8-6be8-47c3-8365-48ba528d4dab None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:46.434382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3360e3-eeaf-4cf5-bdb8-94577093a2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:46.435428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3360e3-eeaf-4cf5-bdb8-94577093a2ff None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.436135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3360e3-eeaf-4cf5-bdb8-94577093a2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.436579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3360e3-eeaf-4cf5-bdb8-94577093a2ff None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.437440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3360e3-eeaf-4cf5-bdb8-94577093a2ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:46.437936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3360e3-eeaf-4cf5-bdb8-94577093a2ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:46.439930 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c3360e3-eeaf-4cf5-bdb8-94577093a2ff None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:46.451893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 242/494] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:29:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:46.453051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3360e3-eeaf-4cf5-bdb8-94577093a2ff None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:46.539039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:46.541781 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1326e8-6be8-47c3-8365-48ba528d4dab None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:46.564960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:46.583772 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1368a86-c5a1-40a4-b5ee-787b20b2644f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:46.584487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1368a86-c5a1-40a4-b5ee-787b20b2644f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.584777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1368a86-c5a1-40a4-b5ee-787b20b2644f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.585046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1368a86-c5a1-40a4-b5ee-787b20b2644f None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.585767 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1368a86-c5a1-40a4-b5ee-787b20b2644f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:46.586062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1368a86-c5a1-40a4-b5ee-787b20b2644f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:46.587239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3360e3-eeaf-4cf5-bdb8-94577093a2ff None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:46.587471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1368a86-c5a1-40a4-b5ee-787b20b2644f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:46.621808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da99bd01-d0a1-4fee-ab7f-b79949cc5192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:46.622413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99bd01-d0a1-4fee-ab7f-b79949cc5192 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.622686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99bd01-d0a1-4fee-ab7f-b79949cc5192 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.622952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da99bd01-d0a1-4fee-ab7f-b79949cc5192 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.623615 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99bd01-d0a1-4fee-ab7f-b79949cc5192 None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:46.623917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99bd01-d0a1-4fee-ab7f-b79949cc5192 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:46.625329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da99bd01-d0a1-4fee-ab7f-b79949cc5192 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:46.628047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 253/495] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Sep 6 22:29:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:46.629702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1368a86-c5a1-40a4-b5ee-787b20b2644f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:46.636787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 243/496] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:29:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:46.642878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986cca3-fcec-484b-9a5e-88e1b4f87a9e None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.643165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986cca3-fcec-484b-9a5e-88e1b4f87a9e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.643437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986cca3-fcec-484b-9a5e-88e1b4f87a9e None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.645031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 244/497] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:46.646321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986cca3-fcec-484b-9a5e-88e1b4f87a9e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:46.732533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1368a86-c5a1-40a4-b5ee-787b20b2644f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:46.749421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986cca3-fcec-484b-9a5e-88e1b4f87a9e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:46.767261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5264e0-0492-4533-b77c-9a1c82f46656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:46.767879 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5264e0-0492-4533-b77c-9a1c82f46656 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.768151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5264e0-0492-4533-b77c-9a1c82f46656 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.768416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5264e0-0492-4533-b77c-9a1c82f46656 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.777795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:46.779059 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5264e0-0492-4533-b77c-9a1c82f46656 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:46.780782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5264e0-0492-4533-b77c-9a1c82f46656 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:46.783985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2b9bf-365e-4831-9e57-60cf04ae86d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:46.784582 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2b9bf-365e-4831-9e57-60cf04ae86d6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.784900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2b9bf-365e-4831-9e57-60cf04ae86d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.785657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2b9bf-365e-4831-9e57-60cf04ae86d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-1d5264e0-0492-4533-b77c-9a1c82f46656 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:46.796148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:46.797114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:46.797963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:46.797963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:46.797963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:46.797963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 254/498] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:46.798175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5264e0-0492-4533-b77c-9a1c82f46656 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:46.828588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:46.859758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2b9bf-365e-4831-9e57-60cf04ae86d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:46.861250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2b9bf-365e-4831-9e57-60cf04ae86d6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:46.902934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5264e0-0492-4533-b77c-9a1c82f46656 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:46.937743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a732d205-051e-4482-85b6-ed52870623a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:46.938358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d205-051e-4482-85b6-ed52870623a3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.938633 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d205-051e-4482-85b6-ed52870623a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.938902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a732d205-051e-4482-85b6-ed52870623a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.952583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732d205-051e-4482-85b6-ed52870623a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:46.954049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a732d205-051e-4482-85b6-ed52870623a3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:46.964935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 245/499] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:46.966160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9449a9-7a19-4453-85d8-be1757209a59 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.966493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9449a9-7a19-4453-85d8-be1757209a59 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.966765 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9449a9-7a19-4453-85d8-be1757209a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-a732d205-051e-4482-85b6-ed52870623a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:46.968805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:46.969655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:46.970480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:46.970480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:46.970480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:46.970480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 255/500] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:46.971082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537a5fb-f530-4ece-84c0-bff10c7a0f18 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:46.971303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537a5fb-f530-4ece-84c0-bff10c7a0f18 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:46.971572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537a5fb-f530-4ece-84c0-bff10c7a0f18 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:46.976825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 256/501] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:46.976825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537a5fb-f530-4ece-84c0-bff10c7a0f18 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:46.990241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-fd9449a9-7a19-4453-85d8-be1757209a59 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:46.991425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fd9449a9-7a19-4453-85d8-be1757209a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:47.014639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:47.056096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:47.057356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 246/502] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:47.058303 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9449a9-7a19-4453-85d8-be1757209a59 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:47.079806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537a5fb-f530-4ece-84c0-bff10c7a0f18 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:47.120013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2dd9a-a611-462c-a7d8-925c077d81d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:47.120640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2dd9a-a611-462c-a7d8-925c077d81d7 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.120955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2dd9a-a611-462c-a7d8-925c077d81d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.121401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a2dd9a-a611-462c-a7d8-925c077d81d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.121853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2dd9a-a611-462c-a7d8-925c077d81d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:47.122154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2dd9a-a611-462c-a7d8-925c077d81d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:47.123511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a2dd9a-a611-462c-a7d8-925c077d81d7 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:47.135686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 257/503] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:47.136463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2dd9a-a611-462c-a7d8-925c077d81d7 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:47.167109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9449a9-7a19-4453-85d8-be1757209a59 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:47.205717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49d3ec6-330e-44ab-ac67-d640bd799b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:47.206350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d3ec6-330e-44ab-ac67-d640bd799b2b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.206615 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d3ec6-330e-44ab-ac67-d640bd799b2b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.206879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49d3ec6-330e-44ab-ac67-d640bd799b2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.217727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d3ec6-330e-44ab-ac67-d640bd799b2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:47.219171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49d3ec6-330e-44ab-ac67-d640bd799b2b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-b49d3ec6-330e-44ab-ac67-d640bd799b2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:47.234894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:47.235893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:47.236725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:47.236725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:47.236725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:47.236725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 247/504] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:47.242274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b10100-f41f-44ee-afd0-d026b2d52c58 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.242557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b10100-f41f-44ee-afd0-d026b2d52c58 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.242833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b10100-f41f-44ee-afd0-d026b2d52c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.245572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a2dd9a-a611-462c-a7d8-925c077d81d7 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:47.261177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-03b10100-f41f-44ee-afd0-d026b2d52c58 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:47.262356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-03b10100-f41f-44ee-afd0-d026b2d52c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:47.270004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:47.282718 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fe97b-2a11-4455-b67e-42baac0734aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:47.283680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fe97b-2a11-4455-b67e-42baac0734aa None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.284123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fe97b-2a11-4455-b67e-42baac0734aa None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.284564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712fe97b-2a11-4455-b67e-42baac0734aa None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.285850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fe97b-2a11-4455-b67e-42baac0734aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:47.286141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fe97b-2a11-4455-b67e-42baac0734aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:47.287715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712fe97b-2a11-4455-b67e-42baac0734aa None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:47.294685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:47.306368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 258/505] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:47.306368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fe97b-2a11-4455-b67e-42baac0734aa None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:47.340728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 248/506] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:47.341586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b10100-f41f-44ee-afd0-d026b2d52c58 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:47.411091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712fe97b-2a11-4455-b67e-42baac0734aa None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:47.442157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b10100-f41f-44ee-afd0-d026b2d52c58 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:47.446811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff59b61-fd98-4510-ba20-f07cd44a27da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:47.447440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff59b61-fd98-4510-ba20-f07cd44a27da None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.447716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff59b61-fd98-4510-ba20-f07cd44a27da None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.447987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff59b61-fd98-4510-ba20-f07cd44a27da None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.455467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff59b61-fd98-4510-ba20-f07cd44a27da None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:47.456942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff59b61-fd98-4510-ba20-f07cd44a27da None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-6ff59b61-fd98-4510-ba20-f07cd44a27da None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:29:47.469511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:47.470404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 259/507] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:47 2022] GET /identity/v3/users/cinder => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:47.471335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff59b61-fd98-4510-ba20-f07cd44a27da None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:47.481458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84114368-6e44-4a23-b988-1507dcba299a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:47.482065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84114368-6e44-4a23-b988-1507dcba299a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.482359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84114368-6e44-4a23-b988-1507dcba299a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.482625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84114368-6e44-4a23-b988-1507dcba299a None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.483258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84114368-6e44-4a23-b988-1507dcba299a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:47.483558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84114368-6e44-4a23-b988-1507dcba299a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:47.484940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84114368-6e44-4a23-b988-1507dcba299a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:47.501126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 249/508] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:47.501126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84114368-6e44-4a23-b988-1507dcba299a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:47.501126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:47.538969 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:47.598472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff59b61-fd98-4510-ba20-f07cd44a27da None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:47.624541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84114368-6e44-4a23-b988-1507dcba299a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:47.631943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bad01-5d63-4d53-8778-366ea1e7efe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:47.632844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bad01-5d63-4d53-8778-366ea1e7efe4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.633123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bad01-5d63-4d53-8778-366ea1e7efe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.633392 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425bad01-5d63-4d53-8778-366ea1e7efe4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.640819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bad01-5d63-4d53-8778-366ea1e7efe4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:47.642252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425bad01-5d63-4d53-8778-366ea1e7efe4 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-425bad01-5d63-4d53-8778-366ea1e7efe4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:29:47.653765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:47.654625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:47.655523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:47.655523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:47.655523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:29:47.655523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:29:47.655523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:29:47.655523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:29:47.655523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:29:47.655523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:29:47.655523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 22:29:47.655523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:47.655523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 260/509] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:47 2022] GET /identity/v3/users/glance => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:47.656079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bad01-5d63-4d53-8778-366ea1e7efe4 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:47.664180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce7e0b-7c1f-4644-b345-c3e22b3a9a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:47.664843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce7e0b-7c1f-4644-b345-c3e22b3a9a3a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.665123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce7e0b-7c1f-4644-b345-c3e22b3a9a3a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.665392 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ce7e0b-7c1f-4644-b345-c3e22b3a9a3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.675683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ce7e0b-7c1f-4644-b345-c3e22b3a9a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-58ce7e0b-7c1f-4644-b345-c3e22b3a9a3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:47.681059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:47.681941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 250/510] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:47.682972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce7e0b-7c1f-4644-b345-c3e22b3a9a3a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:47.742556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:47.759193 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425bad01-5d63-4d53-8778-366ea1e7efe4 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:47.776439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:47.782822 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ce7e0b-7c1f-4644-b345-c3e22b3a9a3a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:47.794484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09bba93-6a17-4fb9-8647-07267caa7e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:47.795086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bba93-6a17-4fb9-8647-07267caa7e48 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.795364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bba93-6a17-4fb9-8647-07267caa7e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.795626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09bba93-6a17-4fb9-8647-07267caa7e48 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.796623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09bba93-6a17-4fb9-8647-07267caa7e48 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:47.796623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09bba93-6a17-4fb9-8647-07267caa7e48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:47.798078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09bba93-6a17-4fb9-8647-07267caa7e48 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:47.815672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 261/511] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Sep 6 22:29:47 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:47.817069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09bba93-6a17-4fb9-8647-07267caa7e48 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:47.821448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c51f7-0a28-4204-a6ac-d59fcd4a0dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:47.822073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c51f7-0a28-4204-a6ac-d59fcd4a0dad None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.822380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c51f7-0a28-4204-a6ac-d59fcd4a0dad None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.822650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c51f7-0a28-4204-a6ac-d59fcd4a0dad None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.829464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c51f7-0a28-4204-a6ac-d59fcd4a0dad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:47.830914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c51f7-0a28-4204-a6ac-d59fcd4a0dad None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-377c51f7-0a28-4204-a6ac-d59fcd4a0dad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:29:47.842684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:47.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:47.844417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:29:47.844417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:29:47.844417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:29:47.844417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:29:47.844417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:29:47.844417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:29:47.844417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:47.844417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:47.844417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 251/512] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:47.844908 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c51f7-0a28-4204-a6ac-d59fcd4a0dad None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:47.924169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09bba93-6a17-4fb9-8647-07267caa7e48 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:47.943964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c51f7-0a28-4204-a6ac-d59fcd4a0dad None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:47.958470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25fc7e-d1f3-47d9-859e-1e88793b2dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:47.959093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25fc7e-d1f3-47d9-859e-1e88793b2dda None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.959372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25fc7e-d1f3-47d9-859e-1e88793b2dda None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.959640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e25fc7e-d1f3-47d9-859e-1e88793b2dda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.965263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:47.969814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25fc7e-d1f3-47d9-859e-1e88793b2dda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:47.971262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e25fc7e-d1f3-47d9-859e-1e88793b2dda None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:47.977350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061c281-b660-4ca0-af75-d89b523f5c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:47.977942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061c281-b660-4ca0-af75-d89b523f5c9f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:47.978215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061c281-b660-4ca0-af75-d89b523f5c9f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:47.978484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e061c281-b660-4ca0-af75-d89b523f5c9f None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:47.979231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061c281-b660-4ca0-af75-d89b523f5c9f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:47.979530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061c281-b660-4ca0-af75-d89b523f5c9f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:47.980951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e061c281-b660-4ca0-af75-d89b523f5c9f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-5e25fc7e-d1f3-47d9-859e-1e88793b2dda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:47.986272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:47.987291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:47.988176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:47.988176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:47.988176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:47.988176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 262/513] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:47.988523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25fc7e-d1f3-47d9-859e-1e88793b2dda None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:48.017526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 252/514] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Sep 6 22:29:47 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:48.018272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061c281-b660-4ca0-af75-d89b523f5c9f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:48.018439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:48.091930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e25fc7e-d1f3-47d9-859e-1e88793b2dda None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:48.116243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e061c281-b660-4ca0-af75-d89b523f5c9f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:48.124988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f5256-de54-4238-9856-691f50eb20d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:48.125592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f5256-de54-4238-9856-691f50eb20d9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:48.125869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f5256-de54-4238-9856-691f50eb20d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:48.126136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142f5256-de54-4238-9856-691f50eb20d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:48.126790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f5256-de54-4238-9856-691f50eb20d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:48.127096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f5256-de54-4238-9856-691f50eb20d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:48.128493 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142f5256-de54-4238-9856-691f50eb20d9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:48.137057 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 263/515] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:29:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:48.141759 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f5256-de54-4238-9856-691f50eb20d9 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:48.150687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fc9d0-cf25-4a0b-b090-ae2a83ae77b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:48.151292 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fc9d0-cf25-4a0b-b090-ae2a83ae77b8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:48.151566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fc9d0-cf25-4a0b-b090-ae2a83ae77b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:48.151829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10fc9d0-cf25-4a0b-b090-ae2a83ae77b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:48.200180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:48.219940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fc9d0-cf25-4a0b-b090-ae2a83ae77b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:48.221446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10fc9d0-cf25-4a0b-b090-ae2a83ae77b8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:48.229620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:48.240008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142f5256-de54-4238-9856-691f50eb20d9 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:48.274389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3827f97-b75e-45dd-ad74-cee9aad3669c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:48.274913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3827f97-b75e-45dd-ad74-cee9aad3669c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:48.275183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3827f97-b75e-45dd-ad74-cee9aad3669c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:48.275448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3827f97-b75e-45dd-ad74-cee9aad3669c None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:48.276118 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3827f97-b75e-45dd-ad74-cee9aad3669c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:48.276413 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3827f97-b75e-45dd-ad74-cee9aad3669c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:48.277861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3827f97-b75e-45dd-ad74-cee9aad3669c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:48.295916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 264/516] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:29:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:48.296715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3827f97-b75e-45dd-ad74-cee9aad3669c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:48.322994 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 253/517] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:48.323820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fc9d0-cf25-4a0b-b090-ae2a83ae77b8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:48.400548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3827f97-b75e-45dd-ad74-cee9aad3669c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:48.401175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:48.422316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10fc9d0-cf25-4a0b-b090-ae2a83ae77b8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:48.433695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca13eb-fe29-41d7-bf39-b08df4053b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:48.434292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca13eb-fe29-41d7-bf39-b08df4053b51 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:48.434568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca13eb-fe29-41d7-bf39-b08df4053b51 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:48.434839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca13eb-fe29-41d7-bf39-b08df4053b51 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:48.435477 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca13eb-fe29-41d7-bf39-b08df4053b51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:48.435775 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca13eb-fe29-41d7-bf39-b08df4053b51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:48.437204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca13eb-fe29-41d7-bf39-b08df4053b51 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:48.448808 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 265/518] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:48.449591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca13eb-fe29-41d7-bf39-b08df4053b51 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:48.456591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ef599-cf33-4cbd-914f-71b1be3a9cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:48.457970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ef599-cf33-4cbd-914f-71b1be3a9cca None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:48.458701 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ef599-cf33-4cbd-914f-71b1be3a9cca None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:48.458978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091ef599-cf33-4cbd-914f-71b1be3a9cca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:48.461659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:48.526428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ef599-cf33-4cbd-914f-71b1be3a9cca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:48.527791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091ef599-cf33-4cbd-914f-71b1be3a9cca None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:48.547513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca13eb-fe29-41d7-bf39-b08df4053b51 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:48.582494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8bbd0-08e1-45b4-9b25-475eb6877cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:48.583154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8bbd0-08e1-45b4-9b25-475eb6877cfd None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:48.583425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8bbd0-08e1-45b4-9b25-475eb6877cfd None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:48.583694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb8bbd0-08e1-45b4-9b25-475eb6877cfd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:48.593687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8bbd0-08e1-45b4-9b25-475eb6877cfd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:48.595112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb8bbd0-08e1-45b4-9b25-475eb6877cfd None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-4eb8bbd0-08e1-45b4-9b25-475eb6877cfd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:48.610617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:48.611555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:48.612398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:48.612398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:48.612398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:48.612398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 266/519] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:48.612621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8bbd0-08e1-45b4-9b25-475eb6877cfd None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:48.613606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:48.636972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 254/520] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:48.638106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ef599-cf33-4cbd-914f-71b1be3a9cca None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:48.674385 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:48.714042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb8bbd0-08e1-45b4-9b25-475eb6877cfd None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:48.734964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091ef599-cf33-4cbd-914f-71b1be3a9cca None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:48.746686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadea82-869f-4de1-ad3e-6a05b7a2ee66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:48.747279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadea82-869f-4de1-ad3e-6a05b7a2ee66 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:48.747554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadea82-869f-4de1-ad3e-6a05b7a2ee66 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:48.747823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadea82-869f-4de1-ad3e-6a05b7a2ee66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:48.748539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadea82-869f-4de1-ad3e-6a05b7a2ee66 None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=b16947313dad4d59a97cc8b016ee54c1) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:48.748882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadea82-869f-4de1-ad3e-6a05b7a2ee66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:48.750281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadea82-869f-4de1-ad3e-6a05b7a2ee66 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:48.756216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 267/521] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:48 2022] GET /identity/v3/role_assignments?user.id=b16947313dad4d59a97cc8b016ee54c1&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:48.756823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadea82-869f-4de1-ad3e-6a05b7a2ee66 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:48.772884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f4f20-358f-4f7a-8713-5a50e14c174b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:48.773584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f4f20-358f-4f7a-8713-5a50e14c174b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:48.773816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f4f20-358f-4f7a-8713-5a50e14c174b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:48.774072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092f4f20-358f-4f7a-8713-5a50e14c174b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:48.785780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f4f20-358f-4f7a-8713-5a50e14c174b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:48.787188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092f4f20-358f-4f7a-8713-5a50e14c174b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-092f4f20-358f-4f7a-8713-5a50e14c174b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:48.804181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:48.805422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:48.806267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:48.806267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:48.806267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:48.806267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 255/522] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:48.807431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f4f20-358f-4f7a-8713-5a50e14c174b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:48.846057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:48.879436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadea82-869f-4de1-ad3e-6a05b7a2ee66 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:48.911097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:48.920815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a576ff3-3f90-4202-b701-b60622a622b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:48.921444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a576ff3-3f90-4202-b701-b60622a622b1 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:48.921725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a576ff3-3f90-4202-b701-b60622a622b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:48.921998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a576ff3-3f90-4202-b701-b60622a622b1 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:48.922653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a576ff3-3f90-4202-b701-b60622a622b1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:48.924126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a576ff3-3f90-4202-b701-b60622a622b1 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:48.933202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092f4f20-358f-4f7a-8713-5a50e14c174b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:48.949520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 268/523] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:29:48 2022] POST /identity/v3/users => generated 247 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:29:48.950496 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a576ff3-3f90-4202-b701-b60622a622b1 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:48.990516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62a222-9157-45c0-8894-8b3333af927a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:48.992200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62a222-9157-45c0-8894-8b3333af927a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:48.992699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62a222-9157-45c0-8894-8b3333af927a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:48.994241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e62a222-9157-45c0-8894-8b3333af927a None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:48.995157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62a222-9157-45c0-8894-8b3333af927a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:48.995660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62a222-9157-45c0-8894-8b3333af927a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:48.997320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e62a222-9157-45c0-8894-8b3333af927a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:49.015426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 256/524] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:49.016776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62a222-9157-45c0-8894-8b3333af927a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:49.077172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a576ff3-3f90-4202-b701-b60622a622b1 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:49.117807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22f46-514f-4fba-b87a-f6dc7944e5ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:49.118728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22f46-514f-4fba-b87a-f6dc7944e5ab None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:49.119145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22f46-514f-4fba-b87a-f6dc7944e5ab None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:49.119825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b22f46-514f-4fba-b87a-f6dc7944e5ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:49.133360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b22f46-514f-4fba-b87a-f6dc7944e5ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:49.135204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b22f46-514f-4fba-b87a-f6dc7944e5ab None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:49.139237 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:49.150893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e62a222-9157-45c0-8894-8b3333af927a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-f7b22f46-514f-4fba-b87a-f6dc7944e5ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:49.152727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:49.153855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:49.154953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:49.154953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:49.154953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:49.156027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 269/525] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:49.161821 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22f46-514f-4fba-b87a-f6dc7944e5ab None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:49.199185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd0688-e9a5-4530-8af5-a76aa5a899cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:49.199800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd0688-e9a5-4530-8af5-a76aa5a899cd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:49.200078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd0688-e9a5-4530-8af5-a76aa5a899cd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:49.200408 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd0688-e9a5-4530-8af5-a76aa5a899cd None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:49.201093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd0688-e9a5-4530-8af5-a76aa5a899cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:49.201431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd0688-e9a5-4530-8af5-a76aa5a899cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:49.202830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd0688-e9a5-4530-8af5-a76aa5a899cd None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:49.203564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:49.218210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 257/526] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:49.218624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd0688-e9a5-4530-8af5-a76aa5a899cd None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:49.288860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b22f46-514f-4fba-b87a-f6dc7944e5ab None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:49.334823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe959a-62f5-4bad-8da9-38305c51e869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:49.335373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe959a-62f5-4bad-8da9-38305c51e869 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:49.335645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe959a-62f5-4bad-8da9-38305c51e869 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:49.335908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fe959a-62f5-4bad-8da9-38305c51e869 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:49.346492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe959a-62f5-4bad-8da9-38305c51e869 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:49.348695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd0688-e9a5-4530-8af5-a76aa5a899cd None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:49.348851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fe959a-62f5-4bad-8da9-38305c51e869 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-f9fe959a-62f5-4bad-8da9-38305c51e869 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:29:49.363054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:49.364014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:49.365621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:49.365621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:49.365621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:29:49.365621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:29:49.365621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:29:49.365621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:29:49.365621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:29:49.365621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:29:49.365621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 22:29:49.365621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:49.365621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 270/527] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:49 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:49.369354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe959a-62f5-4bad-8da9-38305c51e869 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:49.396639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ead10-0c6d-4510-bce0-8c2041297905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:49.397284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ead10-0c6d-4510-bce0-8c2041297905 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:49.397570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ead10-0c6d-4510-bce0-8c2041297905 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:49.397806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5ead10-0c6d-4510-bce0-8c2041297905 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:49.398546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ead10-0c6d-4510-bce0-8c2041297905 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:49.398846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ead10-0c6d-4510-bce0-8c2041297905 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:49.400216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5ead10-0c6d-4510-bce0-8c2041297905 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:49.414032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 258/528] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:49.414959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ead10-0c6d-4510-bce0-8c2041297905 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:49.431314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:49.496038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:49.505870 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fe959a-62f5-4bad-8da9-38305c51e869 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:49.532193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5ead10-0c6d-4510-bce0-8c2041297905 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:49.542196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713df90-874d-4b9b-a5ba-96b186e5eb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:49.543025 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713df90-874d-4b9b-a5ba-96b186e5eb9c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:49.543534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713df90-874d-4b9b-a5ba-96b186e5eb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:49.543994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713df90-874d-4b9b-a5ba-96b186e5eb9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:49.551759 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713df90-874d-4b9b-a5ba-96b186e5eb9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:49.553498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713df90-874d-4b9b-a5ba-96b186e5eb9c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-b713df90-874d-4b9b-a5ba-96b186e5eb9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:29:49.576022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:49.576852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:49.577687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:29:49.577687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:29:49.577687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:29:49.577687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:29:49.577687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:29:49.577687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:29:49.577687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:49.577687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:49.580009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d3e38-bb93-471a-b8a1-38c2af3983c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:49.580633 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d3e38-bb93-471a-b8a1-38c2af3983c3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:49.580935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d3e38-bb93-471a-b8a1-38c2af3983c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:49.584809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089d3e38-bb93-471a-b8a1-38c2af3983c3 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:49.584809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d3e38-bb93-471a-b8a1-38c2af3983c3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:49.584809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d3e38-bb93-471a-b8a1-38c2af3983c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:49.584809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089d3e38-bb93-471a-b8a1-38c2af3983c3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:49.585338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 271/529] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:49.587113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713df90-874d-4b9b-a5ba-96b186e5eb9c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:49.603419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 259/530] 10.222.0.39 () {56 vars in 1139 bytes} [Tue Sep 6 22:29:49 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:49.604367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d3e38-bb93-471a-b8a1-38c2af3983c3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:49.679905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:49.697345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713df90-874d-4b9b-a5ba-96b186e5eb9c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:49.717298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089d3e38-bb93-471a-b8a1-38c2af3983c3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:49.734388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b214f1-6d1d-43a7-9658-4d54cf9effff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:49.735015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b214f1-6d1d-43a7-9658-4d54cf9effff None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:49.735294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b214f1-6d1d-43a7-9658-4d54cf9effff None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:49.735565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b214f1-6d1d-43a7-9658-4d54cf9effff None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:49.743035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b214f1-6d1d-43a7-9658-4d54cf9effff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:49.744469 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b214f1-6d1d-43a7-9658-4d54cf9effff None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:49.755589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32c1bb-4a57-4d90-90f0-c231d81ffed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-93b214f1-6d1d-43a7-9658-4d54cf9effff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:29:49.755911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:49.756910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:49.758020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:29:49.758020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:29:49.758020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:29:49.758020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:29:49.758020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:29:49.758020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:29:49.758020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:49.758020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:49.758430 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32c1bb-4a57-4d90-90f0-c231d81ffed7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:49.758497 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 272/531] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:49.758561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32c1bb-4a57-4d90-90f0-c231d81ffed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:49.758561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32c1bb-4a57-4d90-90f0-c231d81ffed7 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:49.758682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b214f1-6d1d-43a7-9658-4d54cf9effff None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:49.758748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32c1bb-4a57-4d90-90f0-c231d81ffed7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:49.758748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32c1bb-4a57-4d90-90f0-c231d81ffed7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:49.760820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32c1bb-4a57-4d90-90f0-c231d81ffed7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:49.779507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:49.781477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 260/532] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:29:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:49.782580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32c1bb-4a57-4d90-90f0-c231d81ffed7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:49.873084 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b214f1-6d1d-43a7-9658-4d54cf9effff None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:49.895792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32c1bb-4a57-4d90-90f0-c231d81ffed7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:49.909094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6e107-cb37-4ba9-80f2-7b603a1d2139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:49.909935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6e107-cb37-4ba9-80f2-7b603a1d2139 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:49.910347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6e107-cb37-4ba9-80f2-7b603a1d2139 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:49.910892 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c6e107-cb37-4ba9-80f2-7b603a1d2139 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:49.929457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:49.936262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9846d-393a-4e3e-bf96-fedfeedc200c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:49.937611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9846d-393a-4e3e-bf96-fedfeedc200c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:49.938395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9846d-393a-4e3e-bf96-fedfeedc200c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:49.938890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e9846d-393a-4e3e-bf96-fedfeedc200c None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:49.939843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9846d-393a-4e3e-bf96-fedfeedc200c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:49.940330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9846d-393a-4e3e-bf96-fedfeedc200c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:49.942950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e9846d-393a-4e3e-bf96-fedfeedc200c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:49.956212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 261/533] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:29:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:49.957537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9846d-393a-4e3e-bf96-fedfeedc200c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:49.998416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6e107-cb37-4ba9-80f2-7b603a1d2139 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:50.000096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c6e107-cb37-4ba9-80f2-7b603a1d2139 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:50.016787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:50.069587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e9846d-393a-4e3e-bf96-fedfeedc200c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:50.111425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 273/534] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:29:50.111425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6e107-cb37-4ba9-80f2-7b603a1d2139 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:50.114100 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ac3ac-1383-46f9-9d9f-236df5571894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:50.114913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ac3ac-1383-46f9-9d9f-236df5571894 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:50.115386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ac3ac-1383-46f9-9d9f-236df5571894 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:50.115825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3ac3ac-1383-46f9-9d9f-236df5571894 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:50.116724 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ac3ac-1383-46f9-9d9f-236df5571894 None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=d2da9e23277a4de2a61b9727737ab8f5) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:50.117340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ac3ac-1383-46f9-9d9f-236df5571894 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:50.118943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3ac3ac-1383-46f9-9d9f-236df5571894 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:50.125734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 262/535] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:49 2022] GET /identity/v3/role_assignments?user.id=d2da9e23277a4de2a61b9727737ab8f5&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:50.126789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ac3ac-1383-46f9-9d9f-236df5571894 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:50.190236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:50.287743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3ac3ac-1383-46f9-9d9f-236df5571894 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:50.304320 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c6e107-cb37-4ba9-80f2-7b603a1d2139 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:50.328923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:50.345899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae938fb-a22a-43e0-9b7a-df932b4e7497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:50.346999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae938fb-a22a-43e0-9b7a-df932b4e7497 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:50.347579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae938fb-a22a-43e0-9b7a-df932b4e7497 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:50.348073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae938fb-a22a-43e0-9b7a-df932b4e7497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:50.358676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360217f3-e19d-4679-95ad-29dd7f99c3c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:50.359281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360217f3-e19d-4679-95ad-29dd7f99c3c6 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:50.359561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360217f3-e19d-4679-95ad-29dd7f99c3c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:50.359833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360217f3-e19d-4679-95ad-29dd7f99c3c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:50.360657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360217f3-e19d-4679-95ad-29dd7f99c3c6 None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=42cf265450454ff9a27a29093336205e) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:50.360903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360217f3-e19d-4679-95ad-29dd7f99c3c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:50.362047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae938fb-a22a-43e0-9b7a-df932b4e7497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:50.362343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360217f3-e19d-4679-95ad-29dd7f99c3c6 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:50.364105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae938fb-a22a-43e0-9b7a-df932b4e7497 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:50.368963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 274/536] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:50 2022] GET /identity/v3/role_assignments?user.id=42cf265450454ff9a27a29093336205e&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 259 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-1ae938fb-a22a-43e0-9b7a-df932b4e7497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:50.380819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:50.381749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:50.382934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:50.382934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:50.382934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:50.385290 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 263/537] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:50.391204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360217f3-e19d-4679-95ad-29dd7f99c3c6 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:50.518768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360217f3-e19d-4679-95ad-29dd7f99c3c6 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:50.579297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99efae-bbdf-4f59-a489-cf9aaa35ff9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:50.579916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99efae-bbdf-4f59-a489-cf9aaa35ff9e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:50.580191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99efae-bbdf-4f59-a489-cf9aaa35ff9e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:50.580514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a99efae-bbdf-4f59-a489-cf9aaa35ff9e None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:50.583986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99efae-bbdf-4f59-a489-cf9aaa35ff9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:50.584294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99efae-bbdf-4f59-a489-cf9aaa35ff9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:50.585745 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a99efae-bbdf-4f59-a489-cf9aaa35ff9e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:50.602655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 275/538] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:50.608309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae938fb-a22a-43e0-9b7a-df932b4e7497 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:50.743655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae938fb-a22a-43e0-9b7a-df932b4e7497 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:50.751254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:50.779954 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630ec97-15d4-4923-8d67-0714a9dbbbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:50.780950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ec97-15d4-4923-8d67-0714a9dbbbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:50.781425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ec97-15d4-4923-8d67-0714a9dbbbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:50.781910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a630ec97-15d4-4923-8d67-0714a9dbbbf0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:50.789212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630ec97-15d4-4923-8d67-0714a9dbbbf0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:50.790867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a630ec97-15d4-4923-8d67-0714a9dbbbf0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a630ec97-15d4-4923-8d67-0714a9dbbbf0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:29:50.802770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:50.803625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:50.804444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:29:50.804444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:29:50.804444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:29:50.804444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:29:50.804444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:29:50.804444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:29:50.804444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:50.804444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:50.806720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 264/539] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:50.812513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99efae-bbdf-4f59-a489-cf9aaa35ff9e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:50.854678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:50.922425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a99efae-bbdf-4f59-a489-cf9aaa35ff9e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:50.959971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8f3f9-2ade-4d5a-8089-9b12801cae40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:50.960988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8f3f9-2ade-4d5a-8089-9b12801cae40 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:50.961460 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8f3f9-2ade-4d5a-8089-9b12801cae40 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:50.961940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8f3f9-2ade-4d5a-8089-9b12801cae40 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:50.962810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8f3f9-2ade-4d5a-8089-9b12801cae40 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:50.963299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8f3f9-2ade-4d5a-8089-9b12801cae40 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:50.964923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8f3f9-2ade-4d5a-8089-9b12801cae40 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:50.977530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 276/540] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:29:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:50.983669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630ec97-15d4-4923-8d67-0714a9dbbbf0 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:51.096779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a630ec97-15d4-4923-8d67-0714a9dbbbf0 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:51.132921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f658c6a-28c4-4b04-8a51-86bb2bcc0b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:51.133864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f658c6a-28c4-4b04-8a51-86bb2bcc0b93 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:51.134334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f658c6a-28c4-4b04-8a51-86bb2bcc0b93 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:51.134821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f658c6a-28c4-4b04-8a51-86bb2bcc0b93 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:51.135723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f658c6a-28c4-4b04-8a51-86bb2bcc0b93 None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=2658435f74a947f39e207465ebff22d8) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:51.136219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f658c6a-28c4-4b04-8a51-86bb2bcc0b93 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:51.137878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f658c6a-28c4-4b04-8a51-86bb2bcc0b93 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:51.143950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 265/541] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:50 2022] GET /identity/v3/role_assignments?user.id=2658435f74a947f39e207465ebff22d8&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:51.232463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb33e85-bc24-448e-8719-e8734e9467f9 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:51.233426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb33e85-bc24-448e-8719-e8734e9467f9 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:51.234979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb33e85-bc24-448e-8719-e8734e9467f9 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:51.236887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 277/542] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:51.243033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c67bc-8a62-4089-a3e6-15864542a702 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:51.243573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c67bc-8a62-4089-a3e6-15864542a702 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:51.244059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2c67bc-8a62-4089-a3e6-15864542a702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:51.263884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2b2c67bc-8a62-4089-a3e6-15864542a702 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:51.265519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2b2c67bc-8a62-4089-a3e6-15864542a702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:51.293617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:51.341419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 266/543] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:51.545027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca8ec3-a2c7-403b-ad72-bde9f478456b None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:51.545335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca8ec3-a2c7-403b-ad72-bde9f478456b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:51.545499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcca8ec3-a2c7-403b-ad72-bde9f478456b None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:51.547046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 278/544] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:51.553366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bf751-c401-4ec2-974a-95b19a6de84e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:51.554013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bf751-c401-4ec2-974a-95b19a6de84e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:51.554492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117bf751-c401-4ec2-974a-95b19a6de84e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:51.573338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-117bf751-c401-4ec2-974a-95b19a6de84e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:51.574840 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-117bf751-c401-4ec2-974a-95b19a6de84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:51.644162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 267/545] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:29:51.730353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca8ec3-a2c7-403b-ad72-bde9f478456b None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:51.866601 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:51.873526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcca8ec3-a2c7-403b-ad72-bde9f478456b None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:51.918500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c13fa7-51ea-4cf9-8cf1-97ad50a01174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:51.919135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c13fa7-51ea-4cf9-8cf1-97ad50a01174 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:51.919412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c13fa7-51ea-4cf9-8cf1-97ad50a01174 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:51.919682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c13fa7-51ea-4cf9-8cf1-97ad50a01174 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:51.936478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c13fa7-51ea-4cf9-8cf1-97ad50a01174 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-55c13fa7-51ea-4cf9-8cf1-97ad50a01174 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:51.942159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:51.943043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:51.943804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:51.946043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 279/546] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:51.953856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bf751-c401-4ec2-974a-95b19a6de84e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:51.959496 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c13fa7-51ea-4cf9-8cf1-97ad50a01174 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:52.108086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c13fa7-51ea-4cf9-8cf1-97ad50a01174 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:52.128054 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:52.146376 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117bf751-c401-4ec2-974a-95b19a6de84e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:52.149846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75063439-f922-4685-9467-ec9b367f5c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:52.150478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75063439-f922-4685-9467-ec9b367f5c67 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:52.150750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75063439-f922-4685-9467-ec9b367f5c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:52.151017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75063439-f922-4685-9467-ec9b367f5c67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:52.163294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75063439-f922-4685-9467-ec9b367f5c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-75063439-f922-4685-9467-ec9b367f5c67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:52.169092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:52.169957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:52.170757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:52.171374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 280/547] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:52.178049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75063439-f922-4685-9467-ec9b367f5c67 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:52.207807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:52.221357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eff84e-5631-4a5e-b13a-f3fe5ef8417e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:52.221972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eff84e-5631-4a5e-b13a-f3fe5ef8417e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:52.222256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eff84e-5631-4a5e-b13a-f3fe5ef8417e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:52.222555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39eff84e-5631-4a5e-b13a-f3fe5ef8417e None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:52.223349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eff84e-5631-4a5e-b13a-f3fe5ef8417e None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:52.223685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eff84e-5631-4a5e-b13a-f3fe5ef8417e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:52.225044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39eff84e-5631-4a5e-b13a-f3fe5ef8417e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:52.245472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 268/548] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:29:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:52.252598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eff84e-5631-4a5e-b13a-f3fe5ef8417e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:52.361638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75063439-f922-4685-9467-ec9b367f5c67 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:52.408013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39eff84e-5631-4a5e-b13a-f3fe5ef8417e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:52.449162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80a8e-4096-49ae-a795-5dd972e24f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:52.449771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80a8e-4096-49ae-a795-5dd972e24f52 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:52.450048 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80a8e-4096-49ae-a795-5dd972e24f52 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:52.450318 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd80a8e-4096-49ae-a795-5dd972e24f52 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:52.451208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd80a8e-4096-49ae-a795-5dd972e24f52 None admin] RBAC: Adding query filter params (name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:52.451516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd80a8e-4096-49ae-a795-5dd972e24f52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:52.452915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd80a8e-4096-49ae-a795-5dd972e24f52 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:52.457893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6f217-59af-4f57-b39d-945b18aa6d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:52.458525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6f217-59af-4f57-b39d-945b18aa6d8c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:52.459035 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6f217-59af-4f57-b39d-945b18aa6d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:52.459035 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6f217-59af-4f57-b39d-945b18aa6d8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:52.474436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 281/549] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:29:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:52.478206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6f217-59af-4f57-b39d-945b18aa6d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:52.480107 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80a8e-4096-49ae-a795-5dd972e24f52 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:52.484257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6f217-59af-4f57-b39d-945b18aa6d8c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-9ef6f217-59af-4f57-b39d-945b18aa6d8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:52.517528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:52.518661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:52.520327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:52.520327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:52.520327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:52.520327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 269/550] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:52.542151 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6f217-59af-4f57-b39d-945b18aa6d8c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:52.546356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:52.641064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:52.683395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd80a8e-4096-49ae-a795-5dd972e24f52 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:52.714573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6f217-59af-4f57-b39d-945b18aa6d8c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:52.729261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ef81d-02b1-450e-887c-54a4628f9a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:52.730322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef81d-02b1-450e-887c-54a4628f9a31 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:52.731967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef81d-02b1-450e-887c-54a4628f9a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:52.732473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9ef81d-02b1-450e-887c-54a4628f9a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:52.756472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ef81d-02b1-450e-887c-54a4628f9a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:52.758571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9ef81d-02b1-450e-887c-54a4628f9a31 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-bf9ef81d-02b1-450e-887c-54a4628f9a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:52.781114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:52.782078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:52.783091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:52.783091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:52.783091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:52.785786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 282/551] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:52.792357 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ef81d-02b1-450e-887c-54a4628f9a31 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:52.797949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3b72a-0d84-4d6a-af60-b20c76e1231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:52.798581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3b72a-0d84-4d6a-af60-b20c76e1231b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:52.798862 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3b72a-0d84-4d6a-af60-b20c76e1231b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:52.799138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a3b72a-0d84-4d6a-af60-b20c76e1231b None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:52.799845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3b72a-0d84-4d6a-af60-b20c76e1231b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:52.800152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3b72a-0d84-4d6a-af60-b20c76e1231b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:52.806576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a3b72a-0d84-4d6a-af60-b20c76e1231b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:52.821916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 270/552] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:52.829438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3b72a-0d84-4d6a-af60-b20c76e1231b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:52.906711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9ef81d-02b1-450e-887c-54a4628f9a31 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:52.945577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:52.965150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541158e-4e26-41fc-b6a8-74662cb18220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:52.965762 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541158e-4e26-41fc-b6a8-74662cb18220 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:52.968793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541158e-4e26-41fc-b6a8-74662cb18220 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:52.968793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3541158e-4e26-41fc-b6a8-74662cb18220 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:52.968793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541158e-4e26-41fc-b6a8-74662cb18220 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:52.968793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541158e-4e26-41fc-b6a8-74662cb18220 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:52.968793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3541158e-4e26-41fc-b6a8-74662cb18220 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:52.983393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 283/553] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:52.989063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541158e-4e26-41fc-b6a8-74662cb18220 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:52.990139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:53.053837 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a3b72a-0d84-4d6a-af60-b20c76e1231b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:53.091377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfe31b-9594-4e7b-b343-83f18cf28549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:53.092440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe31b-9594-4e7b-b343-83f18cf28549 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.092718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe31b-9594-4e7b-b343-83f18cf28549 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.093031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfe31b-9594-4e7b-b343-83f18cf28549 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.103140 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfe31b-9594-4e7b-b343-83f18cf28549 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:53.104654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5dfe31b-9594-4e7b-b343-83f18cf28549 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-c5dfe31b-9594-4e7b-b343-83f18cf28549 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:29:53.121119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:53.122047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:53.122949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:53.122949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:53.122949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:29:53.122949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:29:53.122949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:29:53.122949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:29:53.122949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:29:53.122949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:29:53.122949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:29:53.122949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:53.122949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 271/554] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Sep 6 22:29:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:53.129998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d60e2-99ea-4cea-883b-6c9f09ee514c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.129998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d60e2-99ea-4cea-883b-6c9f09ee514c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.129998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4d60e2-99ea-4cea-883b-6c9f09ee514c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.129998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 272/555] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:53.129998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d60e2-99ea-4cea-883b-6c9f09ee514c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:53.148670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3541158e-4e26-41fc-b6a8-74662cb18220 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:53.190559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2930b2e2-cd74-4997-855d-bfd4ceec9809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:53.191183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930b2e2-cd74-4997-855d-bfd4ceec9809 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.192086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930b2e2-cd74-4997-855d-bfd4ceec9809 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.192491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2930b2e2-cd74-4997-855d-bfd4ceec9809 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.199351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930b2e2-cd74-4997-855d-bfd4ceec9809 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:53.200715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2930b2e2-cd74-4997-855d-bfd4ceec9809 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-2930b2e2-cd74-4997-855d-bfd4ceec9809 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:29:53.212180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:53.213205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:53.214579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:53.214579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:53.214579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:29:53.214579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:29:53.214579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:29:53.214579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:29:53.214579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:29:53.214579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:29:53.214579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:29:53.214579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:53.218047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 284/556] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:29:52 2022] GET /identity/v3/users/swift => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:53.219079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cd860-c60f-422c-8c50-2967cef950f3 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.219358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cd860-c60f-422c-8c50-2967cef950f3 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.219624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55cd860-c60f-422c-8c50-2967cef950f3 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.220868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 285/557] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:53 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:53.221903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17335852-b7ae-4fca-8d23-dd81d14156e7 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.222240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17335852-b7ae-4fca-8d23-dd81d14156e7 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.223044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17335852-b7ae-4fca-8d23-dd81d14156e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.240796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-17335852-b7ae-4fca-8d23-dd81d14156e7 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:53.241994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-17335852-b7ae-4fca-8d23-dd81d14156e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:53.244445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4d60e2-99ea-4cea-883b-6c9f09ee514c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:53.271304 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:53.284647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdd27f-c55f-44ad-a9b8-8bd8d6957c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:53.285789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdd27f-c55f-44ad-a9b8-8bd8d6957c33 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.286071 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdd27f-c55f-44ad-a9b8-8bd8d6957c33 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.286549 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fdd27f-c55f-44ad-a9b8-8bd8d6957c33 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.287082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdd27f-c55f-44ad-a9b8-8bd8d6957c33 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:53.287383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdd27f-c55f-44ad-a9b8-8bd8d6957c33 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:53.289160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fdd27f-c55f-44ad-a9b8-8bd8d6957c33 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:53.292514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:53.307568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 273/558] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Sep 6 22:29:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:53.308301 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdd27f-c55f-44ad-a9b8-8bd8d6957c33 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:53.319196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 286/559] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:53.323504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d55aa0-253e-4df3-976d-6d0807f7363b None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.324214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d55aa0-253e-4df3-976d-6d0807f7363b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.325200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d55aa0-253e-4df3-976d-6d0807f7363b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.356189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-08d55aa0-253e-4df3-976d-6d0807f7363b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:53.357942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-08d55aa0-253e-4df3-976d-6d0807f7363b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:53.415578 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fdd27f-c55f-44ad-a9b8-8bd8d6957c33 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:53.444147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 287/560] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:53.445652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d55aa0-253e-4df3-976d-6d0807f7363b None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:53.465957 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b013b-a68c-4d36-8d72-f2c156f1f61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:53.466905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b013b-a68c-4d36-8d72-f2c156f1f61e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.467365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b013b-a68c-4d36-8d72-f2c156f1f61e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.467812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350b013b-a68c-4d36-8d72-f2c156f1f61e None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.468710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b013b-a68c-4d36-8d72-f2c156f1f61e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:53.469070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b013b-a68c-4d36-8d72-f2c156f1f61e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:53.470625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350b013b-a68c-4d36-8d72-f2c156f1f61e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:53.507730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 274/561] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Sep 6 22:29:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:53.512632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b013b-a68c-4d36-8d72-f2c156f1f61e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:53.558232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d55aa0-253e-4df3-976d-6d0807f7363b None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:53.572938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:53.591866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:53.601374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca10521-daf7-4b5f-b95e-6572df1352fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:53.602324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca10521-daf7-4b5f-b95e-6572df1352fc None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.602851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca10521-daf7-4b5f-b95e-6572df1352fc None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.603319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca10521-daf7-4b5f-b95e-6572df1352fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.617481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca10521-daf7-4b5f-b95e-6572df1352fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:53.619462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca10521-daf7-4b5f-b95e-6572df1352fc None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-cca10521-daf7-4b5f-b95e-6572df1352fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:53.636609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:53.637619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:53.638498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:53.638498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:53.638498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:53.640851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 288/562] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:53.641840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca10521-daf7-4b5f-b95e-6572df1352fc None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:53.667047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350b013b-a68c-4d36-8d72-f2c156f1f61e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:53.703783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3df0e3-01b7-404e-b4a8-3a4263802574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:53.704197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3df0e3-01b7-404e-b4a8-3a4263802574 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.704469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3df0e3-01b7-404e-b4a8-3a4263802574 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.704733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3df0e3-01b7-404e-b4a8-3a4263802574 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.773695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca10521-daf7-4b5f-b95e-6572df1352fc None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:53.778310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3df0e3-01b7-404e-b4a8-3a4263802574 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:53.779804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3df0e3-01b7-404e-b4a8-3a4263802574 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:53.814331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6147056-2262-498f-8dbe-28c8d2659923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:53.814962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6147056-2262-498f-8dbe-28c8d2659923 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.815239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6147056-2262-498f-8dbe-28c8d2659923 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.815513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6147056-2262-498f-8dbe-28c8d2659923 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.824433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:53.827538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6147056-2262-498f-8dbe-28c8d2659923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-b6147056-2262-498f-8dbe-28c8d2659923 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:53.833175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:53.834083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 289/563] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:53.835502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6147056-2262-498f-8dbe-28c8d2659923 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:53.864577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:53.891169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 275/564] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 22:29:53.891861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3df0e3-01b7-404e-b4a8-3a4263802574 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:53.946094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6147056-2262-498f-8dbe-28c8d2659923 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:53.978691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02298ff3-b5b6-4460-b7ae-f60afe0cb03c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:53.979285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02298ff3-b5b6-4460-b7ae-f60afe0cb03c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:53.979560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02298ff3-b5b6-4460-b7ae-f60afe0cb03c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:53.979827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02298ff3-b5b6-4460-b7ae-f60afe0cb03c None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:53.980458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02298ff3-b5b6-4460-b7ae-f60afe0cb03c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:53.980764 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02298ff3-b5b6-4460-b7ae-f60afe0cb03c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:53.982213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02298ff3-b5b6-4460-b7ae-f60afe0cb03c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:53.992784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3df0e3-01b7-404e-b4a8-3a4263802574 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:53.993644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 290/565] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:53.994416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02298ff3-b5b6-4460-b7ae-f60afe0cb03c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.033065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b9377d-0032-4f00-bc52-c55a77ffee6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.033628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9377d-0032-4f00-bc52-c55a77ffee6e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.033906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9377d-0032-4f00-bc52-c55a77ffee6e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.034181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b9377d-0032-4f00-bc52-c55a77ffee6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.042226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:54.045191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b9377d-0032-4f00-bc52-c55a77ffee6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-89b9377d-0032-4f00-bc52-c55a77ffee6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:54.050326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:54.052520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 276/566] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:54.053555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b883c17-4f23-4fed-a4e3-888206ed740b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.054477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b883c17-4f23-4fed-a4e3-888206ed740b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.054477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b883c17-4f23-4fed-a4e3-888206ed740b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.059557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 277/567] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:54 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:54.059557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b883c17-4f23-4fed-a4e3-888206ed740b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.086303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:54.102587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02298ff3-b5b6-4460-b7ae-f60afe0cb03c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:54.136747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a166ad-a751-4747-aaa8-9a4f1d02d50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.137387 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a166ad-a751-4747-aaa8-9a4f1d02d50f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.137660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a166ad-a751-4747-aaa8-9a4f1d02d50f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.137929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a166ad-a751-4747-aaa8-9a4f1d02d50f None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.138612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a166ad-a751-4747-aaa8-9a4f1d02d50f None admin] RBAC: Adding query filter params (name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:54.138908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a166ad-a751-4747-aaa8-9a4f1d02d50f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:54.140243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a166ad-a751-4747-aaa8-9a4f1d02d50f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:54.148153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 291/568] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:29:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:54.148956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a166ad-a751-4747-aaa8-9a4f1d02d50f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.162776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b883c17-4f23-4fed-a4e3-888206ed740b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:54.197080 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d14b7fbe-1d33-4959-8a9f-57a046ebe266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.197697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b7fbe-1d33-4959-8a9f-57a046ebe266 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.197968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b7fbe-1d33-4959-8a9f-57a046ebe266 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.198230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d14b7fbe-1d33-4959-8a9f-57a046ebe266 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.208314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b7fbe-1d33-4959-8a9f-57a046ebe266 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:54.209773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d14b7fbe-1d33-4959-8a9f-57a046ebe266 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-d14b7fbe-1d33-4959-8a9f-57a046ebe266 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:54.223949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:54.224929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:54.225844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:54.225844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:54.225844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:54.225844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 278/569] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:54.231052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805b287-b879-43bb-aa0e-80ef5c3850cf None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.231052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805b287-b879-43bb-aa0e-80ef5c3850cf None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.231052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4805b287-b879-43bb-aa0e-80ef5c3850cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.245388 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4805b287-b879-43bb-aa0e-80ef5c3850cf None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:54.246563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4805b287-b879-43bb-aa0e-80ef5c3850cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:54.247889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a166ad-a751-4747-aaa8-9a4f1d02d50f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:54.282687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f463dbc-0bb1-497d-a752-564173b05f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.283345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f463dbc-0bb1-497d-a752-564173b05f6d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.283566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f463dbc-0bb1-497d-a752-564173b05f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.283830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f463dbc-0bb1-497d-a752-564173b05f6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.291089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f463dbc-0bb1-497d-a752-564173b05f6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:54.292492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f463dbc-0bb1-497d-a752-564173b05f6d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:54.293764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-1f463dbc-0bb1-497d-a752-564173b05f6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:29:54.304459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:54.305436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:54.306717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:29:54.306717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:29:54.306717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:29:54.306717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:29:54.306717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:29:54.306717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:29:54.306717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:54.306717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:54.306717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 292/570] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:54.306717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f463dbc-0bb1-497d-a752-564173b05f6d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.315348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:54.318700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 279/571] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:54.319526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805b287-b879-43bb-aa0e-80ef5c3850cf None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.413512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f463dbc-0bb1-497d-a752-564173b05f6d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:54.418213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4805b287-b879-43bb-aa0e-80ef5c3850cf None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:54.450720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b630-6ffa-4f62-a3d1-6fd325b3ffa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.451701 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b3edf-b43d-44ea-8842-ecc4e75b7334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.452210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b630-6ffa-4f62-a3d1-6fd325b3ffa3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.452333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b3edf-b43d-44ea-8842-ecc4e75b7334 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.452577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b3edf-b43d-44ea-8842-ecc4e75b7334 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.452870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b3edf-b43d-44ea-8842-ecc4e75b7334 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.453183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b630-6ffa-4f62-a3d1-6fd325b3ffa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.453652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0601b630-6ffa-4f62-a3d1-6fd325b3ffa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.454610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601b630-6ffa-4f62-a3d1-6fd325b3ffa3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:54.455086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601b630-6ffa-4f62-a3d1-6fd325b3ffa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:54.456671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0601b630-6ffa-4f62-a3d1-6fd325b3ffa3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:54.463267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b3edf-b43d-44ea-8842-ecc4e75b7334 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:54.464868 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b3edf-b43d-44ea-8842-ecc4e75b7334 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:54.469645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 293/572] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:29:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:54.469645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b630-6ffa-4f62-a3d1-6fd325b3ffa3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e53b3edf-b43d-44ea-8842-ecc4e75b7334 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:54.484604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:54.485630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:54.486510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:54.486510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:54.486510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:54.486510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 280/573] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:54.486510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b3edf-b43d-44ea-8842-ecc4e75b7334 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.524227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:54.546951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:54.590037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0601b630-6ffa-4f62-a3d1-6fd325b3ffa3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:54.594817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b3edf-b43d-44ea-8842-ecc4e75b7334 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:54.623674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af51d3-3864-4cb5-b85c-d42a52d09d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.624277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af51d3-3864-4cb5-b85c-d42a52d09d46 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.624551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af51d3-3864-4cb5-b85c-d42a52d09d46 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.624850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6af51d3-3864-4cb5-b85c-d42a52d09d46 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.625507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af51d3-3864-4cb5-b85c-d42a52d09d46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:54.625805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af51d3-3864-4cb5-b85c-d42a52d09d46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:54.627166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6af51d3-3864-4cb5-b85c-d42a52d09d46 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:54.627909 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440be9bd-6bf6-4d72-8315-4725c500e8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.628504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440be9bd-6bf6-4d72-8315-4725c500e8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.629633 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440be9bd-6bf6-4d72-8315-4725c500e8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.629914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440be9bd-6bf6-4d72-8315-4725c500e8d9 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.630592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440be9bd-6bf6-4d72-8315-4725c500e8d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:54.630891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440be9bd-6bf6-4d72-8315-4725c500e8d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:54.632536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440be9bd-6bf6-4d72-8315-4725c500e8d9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:54.640188 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 294/574] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:54.641000 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af51d3-3864-4cb5-b85c-d42a52d09d46 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.645671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 281/575] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:29:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:54.646474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440be9bd-6bf6-4d72-8315-4725c500e8d9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.738421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6af51d3-3864-4cb5-b85c-d42a52d09d46 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:54.740260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:54.747252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440be9bd-6bf6-4d72-8315-4725c500e8d9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:54.768565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:54.776718 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959eebe-ff85-456d-99e8-87d04fcd77d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.777361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959eebe-ff85-456d-99e8-87d04fcd77d9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.777635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959eebe-ff85-456d-99e8-87d04fcd77d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.777897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b959eebe-ff85-456d-99e8-87d04fcd77d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.785034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fb35f-4ec6-4db9-b9c8-7565deae0a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.785635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fb35f-4ec6-4db9-b9c8-7565deae0a53 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.785911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fb35f-4ec6-4db9-b9c8-7565deae0a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.786183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217fb35f-4ec6-4db9-b9c8-7565deae0a53 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.786816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fb35f-4ec6-4db9-b9c8-7565deae0a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:54.787112 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fb35f-4ec6-4db9-b9c8-7565deae0a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:54.787995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959eebe-ff85-456d-99e8-87d04fcd77d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:54.788485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217fb35f-4ec6-4db9-b9c8-7565deae0a53 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:54.789439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b959eebe-ff85-456d-99e8-87d04fcd77d9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:54.799431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 282/576] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:54.800209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fb35f-4ec6-4db9-b9c8-7565deae0a53 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-b959eebe-ff85-456d-99e8-87d04fcd77d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:54.804058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:54.805095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:54.806032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:54.806032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:54.806032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:54.806032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 295/577] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:54.806032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959eebe-ff85-456d-99e8-87d04fcd77d9 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.898821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217fb35f-4ec6-4db9-b9c8-7565deae0a53 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:54.905296 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b959eebe-ff85-456d-99e8-87d04fcd77d9 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:54.933866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3a7f9-8ff8-4615-b68d-e085b3b23d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.934489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3a7f9-8ff8-4615-b68d-e085b3b23d7e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.934762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3a7f9-8ff8-4615-b68d-e085b3b23d7e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.935029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3a7f9-8ff8-4615-b68d-e085b3b23d7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.942153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bcf91-5b7c-4c52-86fc-f594acf44547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:54.942709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bcf91-5b7c-4c52-86fc-f594acf44547 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:54.942993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bcf91-5b7c-4c52-86fc-f594acf44547 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:54.943265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8bcf91-5b7c-4c52-86fc-f594acf44547 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:54.945617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d3a7f9-8ff8-4615-b68d-e085b3b23d7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:54.950003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bcf91-5b7c-4c52-86fc-f594acf44547 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-01d3a7f9-8ff8-4615-b68d-e085b3b23d7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:54.950789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:54.951733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:54.953501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:54.954158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8bcf91-5b7c-4c52-86fc-f594acf44547 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:54.954236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 283/578] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:54.954236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3a7f9-8ff8-4615-b68d-e085b3b23d7e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-fd8bcf91-5b7c-4c52-86fc-f594acf44547 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:29:54.964707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:54.965689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:54.966580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:29:54.966580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:29:54.966580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:29:54.966580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:29:54.966580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:29:54.966580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:29:54.966580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:54.966580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:54.966580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 296/579] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:54.966580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bcf91-5b7c-4c52-86fc-f594acf44547 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:54.974955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:55.008697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:55.058665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d3a7f9-8ff8-4615-b68d-e085b3b23d7e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:55.068720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8bcf91-5b7c-4c52-86fc-f594acf44547 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:55.091871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644362a3-8a00-46ad-bfec-c5c9c105a3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:55.092526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644362a3-8a00-46ad-bfec-c5c9c105a3a6 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:55.092833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644362a3-8a00-46ad-bfec-c5c9c105a3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:55.093113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644362a3-8a00-46ad-bfec-c5c9c105a3a6 None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/b16947313dad4d59a97cc8b016ee54c1/roles/29088ad68ec54d74bee2847e2272e4e0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:55.100814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec0d77-ab91-4a9d-afc9-2c2c1090de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:55.101427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec0d77-ab91-4a9d-afc9-2c2c1090de59 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:55.101698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec0d77-ab91-4a9d-afc9-2c2c1090de59 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:55.101961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec0d77-ab91-4a9d-afc9-2c2c1090de59 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:55.107029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644362a3-8a00-46ad-bfec-c5c9c105a3a6 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=b16947313dad4d59a97cc8b016ee54c1, role_id=29088ad68ec54d74bee2847e2272e4e0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:55.108470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644362a3-8a00-46ad-bfec-c5c9c105a3a6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:55.109292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec0d77-ab91-4a9d-afc9-2c2c1090de59 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:55.110685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec0d77-ab91-4a9d-afc9-2c2c1090de59 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:55.121241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 284/580] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:29:54 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/b16947313dad4d59a97cc8b016ee54c1/roles/29088ad68ec54d74bee2847e2272e4e0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:55.122033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644362a3-8a00-46ad-bfec-c5c9c105a3a6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-46ec0d77-ab91-4a9d-afc9-2c2c1090de59 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:29:55.122916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:55.123872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 297/581] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:54 2022] GET /identity/v3/users/cinder => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:55.125474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec0d77-ab91-4a9d-afc9-2c2c1090de59 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:55.197938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:55.245149 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644362a3-8a00-46ad-bfec-c5c9c105a3a6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:55.253758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:55.258640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec0d77-ab91-4a9d-afc9-2c2c1090de59 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:55.286210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09e5a1-8d6a-4a5d-987f-541d443f2599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:55.286834 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09e5a1-8d6a-4a5d-987f-541d443f2599 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:55.287122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09e5a1-8d6a-4a5d-987f-541d443f2599 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:55.287394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09e5a1-8d6a-4a5d-987f-541d443f2599 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:55.288156 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09e5a1-8d6a-4a5d-987f-541d443f2599 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:55.288346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09e5a1-8d6a-4a5d-987f-541d443f2599 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:55.289754 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09e5a1-8d6a-4a5d-987f-541d443f2599 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:55.303742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 285/582] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:55.304785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09e5a1-8d6a-4a5d-987f-541d443f2599 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:55.306846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08bb2b-b02f-4297-809b-96921040a8fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:55.307559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08bb2b-b02f-4297-809b-96921040a8fd None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:55.307965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08bb2b-b02f-4297-809b-96921040a8fd None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:55.308374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c08bb2b-b02f-4297-809b-96921040a8fd None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:55.309312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08bb2b-b02f-4297-809b-96921040a8fd None admin] RBAC: Adding query filter params (name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:55.309555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08bb2b-b02f-4297-809b-96921040a8fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:55.311016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c08bb2b-b02f-4297-809b-96921040a8fd None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:55.321602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 298/583] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:29:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:55.322485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08bb2b-b02f-4297-809b-96921040a8fd None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:55.418871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09e5a1-8d6a-4a5d-987f-541d443f2599 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:55.422737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c08bb2b-b02f-4297-809b-96921040a8fd None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:55.444606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:55.457586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475bd71-f1d0-45d8-b5f2-7b2c10046c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:55.458105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef19426-a925-44f1-9fc2-3b019ef79a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:55.458565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475bd71-f1d0-45d8-b5f2-7b2c10046c37 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:55.458704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef19426-a925-44f1-9fc2-3b019ef79a96 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:55.458867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475bd71-f1d0-45d8-b5f2-7b2c10046c37 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:55.458966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef19426-a925-44f1-9fc2-3b019ef79a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:55.459226 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef19426-a925-44f1-9fc2-3b019ef79a96 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:55.459389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475bd71-f1d0-45d8-b5f2-7b2c10046c37 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:55.459939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef19426-a925-44f1-9fc2-3b019ef79a96 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:55.460236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef19426-a925-44f1-9fc2-3b019ef79a96 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:55.460332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475bd71-f1d0-45d8-b5f2-7b2c10046c37 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:55.460769 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475bd71-f1d0-45d8-b5f2-7b2c10046c37 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:55.461679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef19426-a925-44f1-9fc2-3b019ef79a96 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:55.462446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475bd71-f1d0-45d8-b5f2-7b2c10046c37 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:55.480062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 299/584] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Sep 6 22:29:55 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:55.480696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef19426-a925-44f1-9fc2-3b019ef79a96 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:55.484614 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 286/585] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:29:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:55.485536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475bd71-f1d0-45d8-b5f2-7b2c10046c37 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:55.504867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:55.591474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef19426-a925-44f1-9fc2-3b019ef79a96 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:55.598730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475bd71-f1d0-45d8-b5f2-7b2c10046c37 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:55.631211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8635cf-a734-4568-8991-a36672542a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:55.631731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8635cf-a734-4568-8991-a36672542a92 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:55.632001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8635cf-a734-4568-8991-a36672542a92 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:55.632264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8635cf-a734-4568-8991-a36672542a92 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:55.637441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b889354-624d-4866-8b78-1b7a7976a3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:55.638174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b889354-624d-4866-8b78-1b7a7976a3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:55.638554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b889354-624d-4866-8b78-1b7a7976a3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:55.638928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b889354-624d-4866-8b78-1b7a7976a3d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:55.639757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8635cf-a734-4568-8991-a36672542a92 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:55.641213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8635cf-a734-4568-8991-a36672542a92 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:55.649681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b889354-624d-4866-8b78-1b7a7976a3d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:55.651214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b889354-624d-4866-8b78-1b7a7976a3d3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-9e8635cf-a734-4568-8991-a36672542a92 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:29:55.653069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:55.654162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:55.655088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:55.655088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:55.655088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:29:55.655088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:29:55.655088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:29:55.655088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:29:55.655088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:29:55.655088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:29:55.655088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 22:29:55.655088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:55.655088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 300/586] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:29:55 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:55.655676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8635cf-a734-4568-8991-a36672542a92 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-4b889354-624d-4866-8b78-1b7a7976a3d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:55.673675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:55.674853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:55.677076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:55.677076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:55.677076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:55.681515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 287/587] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:55.682409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b889354-624d-4866-8b78-1b7a7976a3d3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:55.721506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:55.744423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:55.768121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8635cf-a734-4568-8991-a36672542a92 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:55.794827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b889354-624d-4866-8b78-1b7a7976a3d3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:55.804810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515260b-8b22-4fbb-be4f-bb7622dc2ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:55.805840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515260b-8b22-4fbb-be4f-bb7622dc2ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:55.805983 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515260b-8b22-4fbb-be4f-bb7622dc2ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:55.806318 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8515260b-8b22-4fbb-be4f-bb7622dc2ba9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:55.818099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515260b-8b22-4fbb-be4f-bb7622dc2ba9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:55.819735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8515260b-8b22-4fbb-be4f-bb7622dc2ba9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:55.828614 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b71f7-9862-4f38-b602-2008c2740195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:55.829181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b71f7-9862-4f38-b602-2008c2740195 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:55.829461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b71f7-9862-4f38-b602-2008c2740195 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:55.829727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174b71f7-9862-4f38-b602-2008c2740195 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:55.830441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b71f7-9862-4f38-b602-2008c2740195 None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=a3f142a9e141440cbc854ade4f4135a0) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:55.830743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b71f7-9862-4f38-b602-2008c2740195 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:55.832734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174b71f7-9862-4f38-b602-2008c2740195 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-8515260b-8b22-4fbb-be4f-bb7622dc2ba9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:55.836990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:55.838213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:55.839319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:55.839319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:55.839319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:55.844157 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 301/588] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:55.845047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515260b-8b22-4fbb-be4f-bb7622dc2ba9 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:55.845276 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 288/589] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:29:55 2022] GET /identity/v3/role_assignments?user.id=a3f142a9e141440cbc854ade4f4135a0&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:55.846452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b71f7-9862-4f38-b602-2008c2740195 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:55.974570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8515260b-8b22-4fbb-be4f-bb7622dc2ba9 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:55.994569 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:55.998497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174b71f7-9862-4f38-b602-2008c2740195 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:56.013159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:56.024815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550f502-f1d1-4fd9-80f2-926c56fe37ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:56.025864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550f502-f1d1-4fd9-80f2-926c56fe37ee None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:56.026145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550f502-f1d1-4fd9-80f2-926c56fe37ee None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:56.026548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3550f502-f1d1-4fd9-80f2-926c56fe37ee None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:56.027309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550f502-f1d1-4fd9-80f2-926c56fe37ee None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:56.027619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550f502-f1d1-4fd9-80f2-926c56fe37ee None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:56.033982 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3550f502-f1d1-4fd9-80f2-926c56fe37ee None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:56.051172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e890704-abc0-41ce-ba90-eaaaec51256c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:56.052387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e890704-abc0-41ce-ba90-eaaaec51256c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:56.052900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e890704-abc0-41ce-ba90-eaaaec51256c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:56.053380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e890704-abc0-41ce-ba90-eaaaec51256c None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:56.054306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e890704-abc0-41ce-ba90-eaaaec51256c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:56.054818 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e890704-abc0-41ce-ba90-eaaaec51256c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:56.056421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e890704-abc0-41ce-ba90-eaaaec51256c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:56.060843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 302/590] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Sep 6 22:29:55 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:56.061946 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550f502-f1d1-4fd9-80f2-926c56fe37ee None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:56.076752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 289/591] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:56.077955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e890704-abc0-41ce-ba90-eaaaec51256c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:56.175368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3550f502-f1d1-4fd9-80f2-926c56fe37ee None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:56.189144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e890704-abc0-41ce-ba90-eaaaec51256c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:56.211598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b2ed8-2992-4bef-8c93-b6ccd70214e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:56.212203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2ed8-2992-4bef-8c93-b6ccd70214e0 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:56.212477 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2ed8-2992-4bef-8c93-b6ccd70214e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:56.212742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6b2ed8-2992-4bef-8c93-b6ccd70214e0 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:56.213417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b2ed8-2992-4bef-8c93-b6ccd70214e0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:56.213717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b2ed8-2992-4bef-8c93-b6ccd70214e0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:56.215074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6b2ed8-2992-4bef-8c93-b6ccd70214e0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:56.231719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 303/592] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:56.231719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b2ed8-2992-4bef-8c93-b6ccd70214e0 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:56.232335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af884429-df75-4f2c-b3a8-93a49ef7cd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:56.232990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af884429-df75-4f2c-b3a8-93a49ef7cd56 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:56.233264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af884429-df75-4f2c-b3a8-93a49ef7cd56 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:56.233529 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af884429-df75-4f2c-b3a8-93a49ef7cd56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:56.246995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af884429-df75-4f2c-b3a8-93a49ef7cd56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:56.249341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af884429-df75-4f2c-b3a8-93a49ef7cd56 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-af884429-df75-4f2c-b3a8-93a49ef7cd56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:56.269299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:56.270353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:56.271251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:56.271251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:56.271251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:56.272841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 290/593] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:56.273884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af884429-df75-4f2c-b3a8-93a49ef7cd56 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:56.293675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:56.300564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:56.357082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6b2ed8-2992-4bef-8c93-b6ccd70214e0 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:56.409005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af884429-df75-4f2c-b3a8-93a49ef7cd56 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:56.409475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4e21a-4ca3-4fc4-a26f-1587fdf61e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:56.410666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4e21a-4ca3-4fc4-a26f-1587fdf61e48 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:56.410992 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4e21a-4ca3-4fc4-a26f-1587fdf61e48 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:56.411260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a4e21a-4ca3-4fc4-a26f-1587fdf61e48 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:56.425129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4e21a-4ca3-4fc4-a26f-1587fdf61e48 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:56.427703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a4e21a-4ca3-4fc4-a26f-1587fdf61e48 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-d4a4e21a-4ca3-4fc4-a26f-1587fdf61e48 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:29:56.444645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:56.445895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:56.447032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:56.447032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:56.447032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:29:56.447032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:29:56.447032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:29:56.447032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:29:56.447032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:29:56.447032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:29:56.447032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 22:29:56.447032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:56.447934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 304/594] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:56 2022] GET /identity/v3/users/neutron => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:56.448879 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4e21a-4ca3-4fc4-a26f-1587fdf61e48 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:56.451457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a8f07-c0a7-4bd6-9d3f-64c78379b2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:56.452744 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a8f07-c0a7-4bd6-9d3f-64c78379b2de None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:56.453261 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a8f07-c0a7-4bd6-9d3f-64c78379b2de None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:56.453733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312a8f07-c0a7-4bd6-9d3f-64c78379b2de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:56.461645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a8f07-c0a7-4bd6-9d3f-64c78379b2de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:56.463359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312a8f07-c0a7-4bd6-9d3f-64c78379b2de None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-312a8f07-c0a7-4bd6-9d3f-64c78379b2de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:29:56.476007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:56.476928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:56.478066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:29:56.478066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:29:56.478066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:29:56.478066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:29:56.478066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:29:56.478066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:29:56.478066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:56.478066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:56.480323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 291/595] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:56.481334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a8f07-c0a7-4bd6-9d3f-64c78379b2de None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:56.561062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:56.587913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a4e21a-4ca3-4fc4-a26f-1587fdf61e48 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:56.599407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:56.606780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312a8f07-c0a7-4bd6-9d3f-64c78379b2de None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:56.626569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974975c-51a1-43fd-8534-d36faa404554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:56.627348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974975c-51a1-43fd-8534-d36faa404554 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:56.627726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974975c-51a1-43fd-8534-d36faa404554 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:56.628131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c974975c-51a1-43fd-8534-d36faa404554 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:56.629705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974975c-51a1-43fd-8534-d36faa404554 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:56.630175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974975c-51a1-43fd-8534-d36faa404554 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:56.631691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c974975c-51a1-43fd-8534-d36faa404554 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:56.645904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 305/596] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:56.646826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974975c-51a1-43fd-8534-d36faa404554 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:56.654057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e2505-d2e7-4de9-ad13-c90f8fa8d8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:56.654862 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2505-d2e7-4de9-ad13-c90f8fa8d8fc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:56.655294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2505-d2e7-4de9-ad13-c90f8fa8d8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:56.655711 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e2505-d2e7-4de9-ad13-c90f8fa8d8fc None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:56.657143 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e2505-d2e7-4de9-ad13-c90f8fa8d8fc None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:56.657612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e2505-d2e7-4de9-ad13-c90f8fa8d8fc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:56.659646 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329e2505-d2e7-4de9-ad13-c90f8fa8d8fc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:56.686093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 292/597] 10.222.0.39 () {56 vars in 1139 bytes} [Tue Sep 6 22:29:56 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:56.687243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e2505-d2e7-4de9-ad13-c90f8fa8d8fc None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:56.773757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c974975c-51a1-43fd-8534-d36faa404554 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:56.804076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e2505-d2e7-4de9-ad13-c90f8fa8d8fc None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:56.815632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9af4bb-60de-4dcc-9bfc-8a6164638aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:56.816271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9af4bb-60de-4dcc-9bfc-8a6164638aac None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:56.816571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9af4bb-60de-4dcc-9bfc-8a6164638aac None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:56.816864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9af4bb-60de-4dcc-9bfc-8a6164638aac None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:56.817551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9af4bb-60de-4dcc-9bfc-8a6164638aac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:56.817849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9af4bb-60de-4dcc-9bfc-8a6164638aac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:56.819361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9af4bb-60de-4dcc-9bfc-8a6164638aac None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:56.837252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 306/598] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:29:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:56.838241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9af4bb-60de-4dcc-9bfc-8a6164638aac None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:56.842101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:56.849420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aafd29-4c2b-488c-aaa4-e444572f94de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:56.850775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafd29-4c2b-488c-aaa4-e444572f94de None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:56.851205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafd29-4c2b-488c-aaa4-e444572f94de None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:56.851647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafd29-4c2b-488c-aaa4-e444572f94de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:56.862164 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aafd29-4c2b-488c-aaa4-e444572f94de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:56.864445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aafd29-4c2b-488c-aaa4-e444572f94de None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:56.874499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f5aafd29-4c2b-488c-aaa4-e444572f94de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:29:56.883050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:56.884147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:56.885078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:29:56.885078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:29:56.885078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:29:56.885078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:29:56.885078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:29:56.885078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:29:56.885078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:56.885078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:56.887545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 293/599] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:56.888448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aafd29-4c2b-488c-aaa4-e444572f94de None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:56.969109 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9af4bb-60de-4dcc-9bfc-8a6164638aac None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:57.002851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aafd29-4c2b-488c-aaa4-e444572f94de None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:57.006649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41a38e-f2cd-4643-bdf7-032c3aa3f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:57.007720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41a38e-f2cd-4643-bdf7-032c3aa3f4fe None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:57.008143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41a38e-f2cd-4643-bdf7-032c3aa3f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:57.008574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d41a38e-f2cd-4643-bdf7-032c3aa3f4fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:57.021007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41a38e-f2cd-4643-bdf7-032c3aa3f4fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:57.022681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d41a38e-f2cd-4643-bdf7-032c3aa3f4fe None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-3d41a38e-f2cd-4643-bdf7-032c3aa3f4fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:57.038260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:57.039447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:57.040546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:57.040546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:57.040546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:57.043326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7fbcd-73ba-4a88-9001-6e53bccd9fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:57.044254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7fbcd-73ba-4a88-9001-6e53bccd9fef None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:57.044735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7fbcd-73ba-4a88-9001-6e53bccd9fef None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:57.045255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc7fbcd-73ba-4a88-9001-6e53bccd9fef None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/d2da9e23277a4de2a61b9727737ab8f5/roles/29088ad68ec54d74bee2847e2272e4e0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:57.046643 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 307/600] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:57.047493 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41a38e-f2cd-4643-bdf7-032c3aa3f4fe None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:57.062683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7fbcd-73ba-4a88-9001-6e53bccd9fef None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=d2da9e23277a4de2a61b9727737ab8f5, role_id=29088ad68ec54d74bee2847e2272e4e0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:57.064702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc7fbcd-73ba-4a88-9001-6e53bccd9fef None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:57.082903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 294/601] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:29:56 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/d2da9e23277a4de2a61b9727737ab8f5/roles/29088ad68ec54d74bee2847e2272e4e0 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:57.083961 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7fbcd-73ba-4a88-9001-6e53bccd9fef None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:57.139004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:57.209482 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:57.213649 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc7fbcd-73ba-4a88-9001-6e53bccd9fef None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:57.249239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d41a38e-f2cd-4643-bdf7-032c3aa3f4fe None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:57.262537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023deca0-fe1f-46ea-8f78-925d56419af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:57.264092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023deca0-fe1f-46ea-8f78-925d56419af9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:57.265325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023deca0-fe1f-46ea-8f78-925d56419af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:57.265848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023deca0-fe1f-46ea-8f78-925d56419af9 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:57.266782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023deca0-fe1f-46ea-8f78-925d56419af9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:57.267338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023deca0-fe1f-46ea-8f78-925d56419af9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:57.269018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023deca0-fe1f-46ea-8f78-925d56419af9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:57.286486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 295/602] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:57.294726 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023deca0-fe1f-46ea-8f78-925d56419af9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:57.337675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428ec2c3-9d16-4d71-85f8-01904f0d1d7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:57.338306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ec2c3-9d16-4d71-85f8-01904f0d1d7f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:57.338588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ec2c3-9d16-4d71-85f8-01904f0d1d7f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:57.338858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428ec2c3-9d16-4d71-85f8-01904f0d1d7f None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:57.339564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ec2c3-9d16-4d71-85f8-01904f0d1d7f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:57.339868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ec2c3-9d16-4d71-85f8-01904f0d1d7f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:57.341356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428ec2c3-9d16-4d71-85f8-01904f0d1d7f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:57.367561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 308/603] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:29:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:57.372871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e741a-d9b4-4ac1-b58b-17ad9233878c None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:57.373162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e741a-d9b4-4ac1-b58b-17ad9233878c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:57.373439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77e741a-d9b4-4ac1-b58b-17ad9233878c None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:57.379476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 309/604] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:29:57.380485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e741a-d9b4-4ac1-b58b-17ad9233878c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:57.430517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:57.435377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023deca0-fe1f-46ea-8f78-925d56419af9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:57.489536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1e57b6-b789-443d-93da-11e5114ce563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:57.490632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e57b6-b789-443d-93da-11e5114ce563 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:57.490917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e57b6-b789-443d-93da-11e5114ce563 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:57.491192 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1e57b6-b789-443d-93da-11e5114ce563 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:57.501082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e57b6-b789-443d-93da-11e5114ce563 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:57.503094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1e57b6-b789-443d-93da-11e5114ce563 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:57.507898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77e741a-d9b4-4ac1-b58b-17ad9233878c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-8e1e57b6-b789-443d-93da-11e5114ce563 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:29:57.516370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:57.517317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:57.518200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:29:57.518200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:29:57.518200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:29:57.518200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:29:57.518200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:29:57.518200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:29:57.518200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:29:57.518200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:57.518200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 296/605] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:29:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:57.522971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263995f-0295-4868-9432-d7000dd5b459 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:57.522971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263995f-0295-4868-9432-d7000dd5b459 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:57.522971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d263995f-0295-4868-9432-d7000dd5b459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:57.540801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d263995f-0295-4868-9432-d7000dd5b459 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:57.542047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d263995f-0295-4868-9432-d7000dd5b459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:57.558915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b37963-3065-46cd-bc44-60af83945788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:57.559715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b37963-3065-46cd-bc44-60af83945788 None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:57.560033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b37963-3065-46cd-bc44-60af83945788 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:57.560316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b37963-3065-46cd-bc44-60af83945788 None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/42cf265450454ff9a27a29093336205e/roles/29088ad68ec54d74bee2847e2272e4e0` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:57.583067 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:57.589206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b37963-3065-46cd-bc44-60af83945788 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=42cf265450454ff9a27a29093336205e, role_id=29088ad68ec54d74bee2847e2272e4e0)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:57.591819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b37963-3065-46cd-bc44-60af83945788 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:57.616177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 310/606] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:29:57 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/42cf265450454ff9a27a29093336205e/roles/29088ad68ec54d74bee2847e2272e4e0 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:57.618257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b37963-3065-46cd-bc44-60af83945788 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:57.648922 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 297/607] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:57.808836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b37963-3065-46cd-bc44-60af83945788 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:57.855737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca2814-b071-491d-a86d-8a9822a9ed3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:57.856734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca2814-b071-491d-a86d-8a9822a9ed3a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:57.857335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca2814-b071-491d-a86d-8a9822a9ed3a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:57.857803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca2814-b071-491d-a86d-8a9822a9ed3a None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:57.858690 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca2814-b071-491d-a86d-8a9822a9ed3a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:57.859189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca2814-b071-491d-a86d-8a9822a9ed3a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:57.860834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eca2814-b071-491d-a86d-8a9822a9ed3a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:57.877694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 311/608] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:29:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:57.884298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263995f-0295-4868-9432-d7000dd5b459 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:58.001359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:58.013351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d263995f-0295-4868-9432-d7000dd5b459 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:58.046293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca2814-b071-491d-a86d-8a9822a9ed3a None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:58.048573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82550b2-954f-4187-be73-389de1ffd3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:58.049375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82550b2-954f-4187-be73-389de1ffd3c4 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:58.049650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82550b2-954f-4187-be73-389de1ffd3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:58.049914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82550b2-954f-4187-be73-389de1ffd3c4 None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/2658435f74a947f39e207465ebff22d8/roles/29088ad68ec54d74bee2847e2272e4e0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:58.065976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82550b2-954f-4187-be73-389de1ffd3c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=2658435f74a947f39e207465ebff22d8, role_id=29088ad68ec54d74bee2847e2272e4e0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:58.067874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82550b2-954f-4187-be73-389de1ffd3c4 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:58.093564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 298/609] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:29:57 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/2658435f74a947f39e207465ebff22d8/roles/29088ad68ec54d74bee2847e2272e4e0 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:29:58.095676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6967-eb1f-49d2-8a43-50df327af5ea None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:58.095956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6967-eb1f-49d2-8a43-50df327af5ea None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:58.096281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ae6967-eb1f-49d2-8a43-50df327af5ea None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:58.097842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 299/610] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:58.104543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18872d3b-722b-47a1-91b2-36ec28418eb5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:58.104886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18872d3b-722b-47a1-91b2-36ec28418eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:58.105159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18872d3b-722b-47a1-91b2-36ec28418eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:58.126828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-18872d3b-722b-47a1-91b2-36ec28418eb5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:58.128522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-18872d3b-722b-47a1-91b2-36ec28418eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:58.131647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:58.192985 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eca2814-b071-491d-a86d-8a9822a9ed3a None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:58.216320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 300/611] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:29:58.229713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e678e-99c8-46d7-88c8-4d82863b089b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:58.230399 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e678e-99c8-46d7-88c8-4d82863b089b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:58.230670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e678e-99c8-46d7-88c8-4d82863b089b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:58.230938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6e678e-99c8-46d7-88c8-4d82863b089b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:58.243166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6e678e-99c8-46d7-88c8-4d82863b089b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-4b6e678e-99c8-46d7-88c8-4d82863b089b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:58.249018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:58.250014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:58.251115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 312/612] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:58.257287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18872d3b-722b-47a1-91b2-36ec28418eb5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:58.369563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:58.412535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18872d3b-722b-47a1-91b2-36ec28418eb5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:58.460652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ab5e2-13fa-4303-9ded-c6ad9fef1f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:58.461636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ab5e2-13fa-4303-9ded-c6ad9fef1f1f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:58.462045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ab5e2-13fa-4303-9ded-c6ad9fef1f1f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:58.462446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4ab5e2-13fa-4303-9ded-c6ad9fef1f1f None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:58.463489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ab5e2-13fa-4303-9ded-c6ad9fef1f1f None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:58.463974 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ab5e2-13fa-4303-9ded-c6ad9fef1f1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:58.465812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4ab5e2-13fa-4303-9ded-c6ad9fef1f1f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:58.475604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 301/613] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:29:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:58.480903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e678e-99c8-46d7-88c8-4d82863b089b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:58.557111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ab5e2-13fa-4303-9ded-c6ad9fef1f1f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:58.608862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6e678e-99c8-46d7-88c8-4d82863b089b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:58.622783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:58.648782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed3100-0174-4c04-8645-3fd1d79611b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:58.649638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed3100-0174-4c04-8645-3fd1d79611b2 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:58.650054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed3100-0174-4c04-8645-3fd1d79611b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:58.650513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ed3100-0174-4c04-8645-3fd1d79611b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:58.669559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed3100-0174-4c04-8645-3fd1d79611b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:58.671219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ed3100-0174-4c04-8645-3fd1d79611b2 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-b4ed3100-0174-4c04-8645-3fd1d79611b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:58.700235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:58.701461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:58.702369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:58.702369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:58.702369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:58.702369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 313/614] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:58.708462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed3100-0174-4c04-8645-3fd1d79611b2 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:58.721535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4ab5e2-13fa-4303-9ded-c6ad9fef1f1f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:58.757124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:58.765017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10216265-9a88-430d-b288-3739a7addcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:58.765665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10216265-9a88-430d-b288-3739a7addcb2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:58.765939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10216265-9a88-430d-b288-3739a7addcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:58.766208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10216265-9a88-430d-b288-3739a7addcb2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:58.791473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10216265-9a88-430d-b288-3739a7addcb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-10216265-9a88-430d-b288-3739a7addcb2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:29:58.805854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:58.807044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:58.808023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 302/615] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:29:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:58.813640 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10216265-9a88-430d-b288-3739a7addcb2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:58.857728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ed3100-0174-4c04-8645-3fd1d79611b2 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:58.928409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235db5d5-aa92-426c-bf76-a72fc67e53ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:58.929373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235db5d5-aa92-426c-bf76-a72fc67e53ee None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:58.929663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235db5d5-aa92-426c-bf76-a72fc67e53ee None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:58.929933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235db5d5-aa92-426c-bf76-a72fc67e53ee None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:58.930853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235db5d5-aa92-426c-bf76-a72fc67e53ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:58.931152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235db5d5-aa92-426c-bf76-a72fc67e53ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:58.932696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235db5d5-aa92-426c-bf76-a72fc67e53ee None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:58.956582 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 314/616] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:58.963181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235db5d5-aa92-426c-bf76-a72fc67e53ee None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:58.982617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10216265-9a88-430d-b288-3739a7addcb2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:59.010968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:59.039697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1b07f-0cd1-46f3-b68f-549c3c244e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:59.041682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1b07f-0cd1-46f3-b68f-549c3c244e8d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:59.042119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1b07f-0cd1-46f3-b68f-549c3c244e8d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:59.042539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c1b07f-0cd1-46f3-b68f-549c3c244e8d None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:59.044458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1b07f-0cd1-46f3-b68f-549c3c244e8d None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:59.044941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1b07f-0cd1-46f3-b68f-549c3c244e8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:59.046531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c1b07f-0cd1-46f3-b68f-549c3c244e8d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:59.064902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 303/617] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:29:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:59.070988 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1b07f-0cd1-46f3-b68f-549c3c244e8d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:59.165358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235db5d5-aa92-426c-bf76-a72fc67e53ee None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:59.185188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:59.220310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c05815-767c-47f5-88ca-eabfe48a8602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:59.220960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c05815-767c-47f5-88ca-eabfe48a8602 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:59.221234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c05815-767c-47f5-88ca-eabfe48a8602 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:59.221501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c05815-767c-47f5-88ca-eabfe48a8602 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:59.229476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c05815-767c-47f5-88ca-eabfe48a8602 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:59.230962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c05815-767c-47f5-88ca-eabfe48a8602 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-52c05815-767c-47f5-88ca-eabfe48a8602 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:29:59.243594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:59.244527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:59.245570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:59.245570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:59.245570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:29:59.245570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:29:59.245570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:29:59.245570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:29:59.245570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:29:59.245570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:29:59.245570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:29:59.245570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:29:59.249097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 315/618] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Sep 6 22:29:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:59.250565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c05815-767c-47f5-88ca-eabfe48a8602 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:59.250883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c1b07f-0cd1-46f3-b68f-549c3c244e8d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:59.301850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab98c0-e243-45a0-8df9-df293c2becb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:59.302496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab98c0-e243-45a0-8df9-df293c2becb5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:59.302776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab98c0-e243-45a0-8df9-df293c2becb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:59.303048 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab98c0-e243-45a0-8df9-df293c2becb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:59.328476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab98c0-e243-45a0-8df9-df293c2becb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:59.329978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab98c0-e243-45a0-8df9-df293c2becb5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:59.345192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a4ab98c0-e243-45a0-8df9-df293c2becb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:29:59.359624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:59.360788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:29:59.362935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:29:59.362935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:29:59.362935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:59.362935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 304/619] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:29:59.364640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c05815-767c-47f5-88ca-eabfe48a8602 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:59.369364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab98c0-e243-45a0-8df9-df293c2becb5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:59.401762 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0e47b-ef56-4074-bb5a-641a1425de08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:59.402368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0e47b-ef56-4074-bb5a-641a1425de08 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:59.402692 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0e47b-ef56-4074-bb5a-641a1425de08 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:59.402965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0e47b-ef56-4074-bb5a-641a1425de08 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:59.403782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0e47b-ef56-4074-bb5a-641a1425de08 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:59.404084 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0e47b-ef56-4074-bb5a-641a1425de08 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:59.405590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0e47b-ef56-4074-bb5a-641a1425de08 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:59.424264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 316/620] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Sep 6 22:29:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:29:59.429824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0e47b-ef56-4074-bb5a-641a1425de08 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:59.535860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0e47b-ef56-4074-bb5a-641a1425de08 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:59.555732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab98c0-e243-45a0-8df9-df293c2becb5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:59.571407 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:59.573503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886bc568-4e53-4aa8-9ce9-e4ea63b6a1cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:59.574122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bc568-4e53-4aa8-9ce9-e4ea63b6a1cd None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:59.574503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bc568-4e53-4aa8-9ce9-e4ea63b6a1cd None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:59.574776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886bc568-4e53-4aa8-9ce9-e4ea63b6a1cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:59.588110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:59.619434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869bf49-892d-4870-9913-f63381683ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:59.620057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869bf49-892d-4870-9913-f63381683ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:59.620634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869bf49-892d-4870-9913-f63381683ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:59.620634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f869bf49-892d-4870-9913-f63381683ae8 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:59.621277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869bf49-892d-4870-9913-f63381683ae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:29:59.621577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869bf49-892d-4870-9913-f63381683ae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:59.623248 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f869bf49-892d-4870-9913-f63381683ae8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:59.652792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 305/621] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:29:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:29:59.655099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bc568-4e53-4aa8-9ce9-e4ea63b6a1cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:59.657043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886bc568-4e53-4aa8-9ce9-e4ea63b6a1cd None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:59.659110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869bf49-892d-4870-9913-f63381683ae8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:59.809497 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 317/622] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:29:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 06 22:29:59.812854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b6f4-78aa-465a-b098-d64ddabbe40c None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:59.815535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b6f4-78aa-465a-b098-d64ddabbe40c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:59.815819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b6f4-78aa-465a-b098-d64ddabbe40c None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:59.817518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 318/623] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:29:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:29:59.818243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836b6f4-78aa-465a-b098-d64ddabbe40c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:29:59.854216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f869bf49-892d-4870-9913-f63381683ae8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:59.884395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:29:59.895816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8da8836-eda0-4a13-859c-7df597fbcc15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:29:59.896517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da8836-eda0-4a13-859c-7df597fbcc15 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:59.896886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da8836-eda0-4a13-859c-7df597fbcc15 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:59.897222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8da8836-eda0-4a13-859c-7df597fbcc15 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:59.904925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da8836-eda0-4a13-859c-7df597fbcc15 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:29:59.906635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8da8836-eda0-4a13-859c-7df597fbcc15 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a8da8836-eda0-4a13-859c-7df597fbcc15 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:29:59.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:29:59.920409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:29:59.921429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:29:59.921429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:29:59.921429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:29:59.921429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:29:59.921429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:29:59.921429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:29:59.921429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:29:59.921429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:29:59.921429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:29:59.921429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:29:59.926240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 306/624] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:29:59 2022] GET /identity/v3/users/swift => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:29:59.927692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729f415-c5c1-4699-92f2-c5a515390c2e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:29:59.928038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729f415-c5c1-4699-92f2-c5a515390c2e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:29:59.928377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729f415-c5c1-4699-92f2-c5a515390c2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:29:59.951727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2729f415-c5c1-4699-92f2-c5a515390c2e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:29:59.953649 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2729f415-c5c1-4699-92f2-c5a515390c2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:29:59.957412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836b6f4-78aa-465a-b098-d64ddabbe40c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:29:59.980419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:00.004528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70d95c-4a81-46c5-b938-9c3a7531a1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:00.005561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70d95c-4a81-46c5-b938-9c3a7531a1b9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.006046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70d95c-4a81-46c5-b938-9c3a7531a1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.007034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70d95c-4a81-46c5-b938-9c3a7531a1b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.020918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70d95c-4a81-46c5-b938-9c3a7531a1b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:00.023159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70d95c-4a81-46c5-b938-9c3a7531a1b9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-6c70d95c-4a81-46c5-b938-9c3a7531a1b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:00.042412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:00.043407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:00.044311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:00.044311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:00.044311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:00.044934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 307/625] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:29:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:00.045851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729f415-c5c1-4699-92f2-c5a515390c2e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:00.046744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 319/626] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:29:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:00.057754 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70d95c-4a81-46c5-b938-9c3a7531a1b9 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:00.205309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:00.213644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70d95c-4a81-46c5-b938-9c3a7531a1b9 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:00.241409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729f415-c5c1-4699-92f2-c5a515390c2e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:00.283276 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1866e-04ba-4377-9a01-97758de9b815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:00.284260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1866e-04ba-4377-9a01-97758de9b815 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.284740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1866e-04ba-4377-9a01-97758de9b815 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.285865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1866e-04ba-4377-9a01-97758de9b815 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.286704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1866e-04ba-4377-9a01-97758de9b815 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:00.287360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1866e-04ba-4377-9a01-97758de9b815 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:00.289790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d1866e-04ba-4377-9a01-97758de9b815 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:00.308005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b0ca6-6a15-4e00-8820-f146979d3e8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:00.308966 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b0ca6-6a15-4e00-8820-f146979d3e8c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.309467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b0ca6-6a15-4e00-8820-f146979d3e8c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.309961 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8b0ca6-6a15-4e00-8820-f146979d3e8c None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.310883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b0ca6-6a15-4e00-8820-f146979d3e8c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:00.311554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b0ca6-6a15-4e00-8820-f146979d3e8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:00.313575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 320/627] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:00.315202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8b0ca6-6a15-4e00-8820-f146979d3e8c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:00.319474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1866e-04ba-4377-9a01-97758de9b815 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:00.345130 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 308/628] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Sep 6 22:30:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:00.353930 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b0ca6-6a15-4e00-8820-f146979d3e8c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:00.384402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:00.459623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1866e-04ba-4377-9a01-97758de9b815 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:00.495778 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8b0ca6-6a15-4e00-8820-f146979d3e8c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:00.504416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c8fd5-c558-4f44-a7cf-9998b16e32a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:00.505112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c8fd5-c558-4f44-a7cf-9998b16e32a0 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.505917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c8fd5-c558-4f44-a7cf-9998b16e32a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.506190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c8fd5-c558-4f44-a7cf-9998b16e32a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.515868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c8fd5-c558-4f44-a7cf-9998b16e32a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:00.517747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42c8fd5-c558-4f44-a7cf-9998b16e32a0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-f42c8fd5-c558-4f44-a7cf-9998b16e32a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:00.538359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:00.539345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:00.540244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:00.540244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:00.540244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:00.540244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:00.540244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:00.540244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:00.540244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:00.540244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:00.540690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cf172-e53d-4bda-8d7e-2b7e3ebd0a7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:00.540690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf172-e53d-4bda-8d7e-2b7e3ebd0a7b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.540690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf172-e53d-4bda-8d7e-2b7e3ebd0a7b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.540690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7cf172-e53d-4bda-8d7e-2b7e3ebd0a7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.541424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 321/629] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:00.544804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c8fd5-c558-4f44-a7cf-9998b16e32a0 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:00.564819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cf172-e53d-4bda-8d7e-2b7e3ebd0a7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:00.564819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7cf172-e53d-4bda-8d7e-2b7e3ebd0a7b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:00.567180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-ef7cf172-e53d-4bda-8d7e-2b7e3ebd0a7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:00.585263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:00.586268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:00.591938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:00.591938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:00.591938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:00.591938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 309/630] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:00.591938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cf172-e53d-4bda-8d7e-2b7e3ebd0a7b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:00.661601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:00.671848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c8fd5-c558-4f44-a7cf-9998b16e32a0 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:00.698777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7cf172-e53d-4bda-8d7e-2b7e3ebd0a7b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:00.706836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53a68a-fc6a-48f9-99fd-3f98beb6c337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:00.707421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53a68a-fc6a-48f9-99fd-3f98beb6c337 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.707692 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53a68a-fc6a-48f9-99fd-3f98beb6c337 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.707955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53a68a-fc6a-48f9-99fd-3f98beb6c337 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.717804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53a68a-fc6a-48f9-99fd-3f98beb6c337 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-cb53a68a-fc6a-48f9-99fd-3f98beb6c337 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:00.723082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:00.724142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:00.725037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 322/631] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:30:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:00.725744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53a68a-fc6a-48f9-99fd-3f98beb6c337 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:00.735511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61adf625-0da8-4cf1-b637-eeb3941eb91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:00.736114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adf625-0da8-4cf1-b637-eeb3941eb91d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.736387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adf625-0da8-4cf1-b637-eeb3941eb91d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.736677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61adf625-0da8-4cf1-b637-eeb3941eb91d None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.737375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adf625-0da8-4cf1-b637-eeb3941eb91d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:00.737675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adf625-0da8-4cf1-b637-eeb3941eb91d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:00.739657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61adf625-0da8-4cf1-b637-eeb3941eb91d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:00.751381 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 310/632] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:00.752528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb540f9-16ff-4a65-855d-a2d759a9690b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.752830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb540f9-16ff-4a65-855d-a2d759a9690b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.753256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb540f9-16ff-4a65-855d-a2d759a9690b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.756826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 311/633] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:00.756826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a9c2e-ace9-4afc-8cf5-d4027587f991 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.756826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a9c2e-ace9-4afc-8cf5-d4027587f991 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.756826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862a9c2e-ace9-4afc-8cf5-d4027587f991 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.757216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 312/634] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:00.758175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a9c2e-ace9-4afc-8cf5-d4027587f991 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:00.804406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:00.828780 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53a68a-fc6a-48f9-99fd-3f98beb6c337 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:00.860703 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862a9c2e-ace9-4afc-8cf5-d4027587f991 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:00.862599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d971b-b921-4a7c-b21a-24cdf9073c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:00.864079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d971b-b921-4a7c-b21a-24cdf9073c57 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.864354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d971b-b921-4a7c-b21a-24cdf9073c57 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.864623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205d971b-b921-4a7c-b21a-24cdf9073c57 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.865281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d971b-b921-4a7c-b21a-24cdf9073c57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:00.865577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d971b-b921-4a7c-b21a-24cdf9073c57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:00.866937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205d971b-b921-4a7c-b21a-24cdf9073c57 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:00.879253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 323/635] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:00.879905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d971b-b921-4a7c-b21a-24cdf9073c57 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:00.890029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:00.899924 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009285ec-9fbf-4047-ae44-5a4b81cf97bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:00.900530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009285ec-9fbf-4047-ae44-5a4b81cf97bd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.900836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009285ec-9fbf-4047-ae44-5a4b81cf97bd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.901110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009285ec-9fbf-4047-ae44-5a4b81cf97bd None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.901765 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009285ec-9fbf-4047-ae44-5a4b81cf97bd None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:00.902068 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009285ec-9fbf-4047-ae44-5a4b81cf97bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:00.903443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009285ec-9fbf-4047-ae44-5a4b81cf97bd None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:00.911789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 313/636] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:30:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:00.912952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551add8e-1625-49d3-9ced-c683ce7d5a37 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:00.913234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551add8e-1625-49d3-9ced-c683ce7d5a37 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:00.913516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551add8e-1625-49d3-9ced-c683ce7d5a37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:00.932046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-551add8e-1625-49d3-9ced-c683ce7d5a37 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:00.933280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-551add8e-1625-49d3-9ced-c683ce7d5a37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:00.978510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205d971b-b921-4a7c-b21a-24cdf9073c57 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:01.000085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 314/637] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:01.001153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb034b57-6bd0-4fb3-874c-bb19107c8efa None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:01.001439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb034b57-6bd0-4fb3-874c-bb19107c8efa None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:01.001712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb034b57-6bd0-4fb3-874c-bb19107c8efa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:01.023699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-cb034b57-6bd0-4fb3-874c-bb19107c8efa None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:01.024906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cb034b57-6bd0-4fb3-874c-bb19107c8efa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:01.025514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d948d2-17d4-4471-817b-965d0970492e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:01.026281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d948d2-17d4-4471-817b-965d0970492e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:01.026867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d948d2-17d4-4471-817b-965d0970492e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:01.027282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d948d2-17d4-4471-817b-965d0970492e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:01.028939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d948d2-17d4-4471-817b-965d0970492e None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=b16947313dad4d59a97cc8b016ee54c1) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:01.029396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d948d2-17d4-4471-817b-965d0970492e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:01.030947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d948d2-17d4-4471-817b-965d0970492e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:01.033456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:01.041316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 324/638] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:00 2022] GET /identity/v3/role_assignments?user.id=b16947313dad4d59a97cc8b016ee54c1&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:01.042524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d948d2-17d4-4471-817b-965d0970492e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:01.112330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 315/639] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:01.113688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb034b57-6bd0-4fb3-874c-bb19107c8efa None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:01.183636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:01.203267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d948d2-17d4-4471-817b-965d0970492e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:01.249870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb034b57-6bd0-4fb3-874c-bb19107c8efa None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:01.274485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e3766-ba37-4396-866f-aadf5d1e1136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:01.275112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e3766-ba37-4396-866f-aadf5d1e1136 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:01.275390 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e3766-ba37-4396-866f-aadf5d1e1136 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:01.275663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850e3766-ba37-4396-866f-aadf5d1e1136 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:01.283051 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e3766-ba37-4396-866f-aadf5d1e1136 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:01.284503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850e3766-ba37-4396-866f-aadf5d1e1136 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-850e3766-ba37-4396-866f-aadf5d1e1136 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:01.298408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:01.299420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:01.300336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:01.300336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:01.300336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:01.300336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:01.300336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:01.300336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:01.300336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:01.300336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:01.303142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 325/640] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:01.303698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c557f2-a611-47a0-b4f7-344844c1e861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:01.304271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e3766-ba37-4396-866f-aadf5d1e1136 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:01.304536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c557f2-a611-47a0-b4f7-344844c1e861 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:01.304961 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c557f2-a611-47a0-b4f7-344844c1e861 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:01.305495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c557f2-a611-47a0-b4f7-344844c1e861 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:01.319466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c557f2-a611-47a0-b4f7-344844c1e861 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:01.321488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c557f2-a611-47a0-b4f7-344844c1e861 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-b9c557f2-a611-47a0-b4f7-344844c1e861 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:01.340945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:01.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:01.342798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:01.342798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:01.342798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:01.342798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 316/641] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:01.343062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c557f2-a611-47a0-b4f7-344844c1e861 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:01.384077 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:01.438135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850e3766-ba37-4396-866f-aadf5d1e1136 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:01.465389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:01.468583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c557f2-a611-47a0-b4f7-344844c1e861 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:01.481293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150b0b9-8ec8-4510-8acc-fbdd24977cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:01.481920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150b0b9-8ec8-4510-8acc-fbdd24977cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:01.482197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150b0b9-8ec8-4510-8acc-fbdd24977cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:01.482471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9150b0b9-8ec8-4510-8acc-fbdd24977cb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:01.494794 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9150b0b9-8ec8-4510-8acc-fbdd24977cb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-9150b0b9-8ec8-4510-8acc-fbdd24977cb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:01.500023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:01.501234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 326/642] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:30:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:01.502102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150b0b9-8ec8-4510-8acc-fbdd24977cb6 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:01.517100 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e07f7d-f1f2-42e6-ad31-b95464937484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:01.517717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e07f7d-f1f2-42e6-ad31-b95464937484 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:01.517995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e07f7d-f1f2-42e6-ad31-b95464937484 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:01.518264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e07f7d-f1f2-42e6-ad31-b95464937484 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:01.518958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e07f7d-f1f2-42e6-ad31-b95464937484 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:01.519303 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e07f7d-f1f2-42e6-ad31-b95464937484 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:01.521012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e07f7d-f1f2-42e6-ad31-b95464937484 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:01.533561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 317/643] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:01.534639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e07f7d-f1f2-42e6-ad31-b95464937484 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:01.621732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9150b0b9-8ec8-4510-8acc-fbdd24977cb6 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:01.641062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:01.644671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e07f7d-f1f2-42e6-ad31-b95464937484 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:01.659174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d54f6-9c24-428a-8f6b-772292eebd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:01.659783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d54f6-9c24-428a-8f6b-772292eebd84 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:01.660651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d54f6-9c24-428a-8f6b-772292eebd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:01.661053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d54f6-9c24-428a-8f6b-772292eebd84 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:01.661698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d54f6-9c24-428a-8f6b-772292eebd84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:01.662003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d54f6-9c24-428a-8f6b-772292eebd84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:01.663353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d54f6-9c24-428a-8f6b-772292eebd84 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:01.675636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 327/644] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:01.676641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d54f6-9c24-428a-8f6b-772292eebd84 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:01.688830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2543d-c35f-4662-9f62-e1d169abff4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:01.689662 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2543d-c35f-4662-9f62-e1d169abff4b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:01.690076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2543d-c35f-4662-9f62-e1d169abff4b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:01.690483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee2543d-c35f-4662-9f62-e1d169abff4b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:01.703482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee2543d-c35f-4662-9f62-e1d169abff4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-9ee2543d-c35f-4662-9f62-e1d169abff4b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:01.708917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:01.710033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:01.711208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:01.718013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 318/645] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:30:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:01.718993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2543d-c35f-4662-9f62-e1d169abff4b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:01.742626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:01.782978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d54f6-9c24-428a-8f6b-772292eebd84 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:01.820733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a65d1a-441b-4dd1-b3c5-0320c1b487ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:01.821357 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65d1a-441b-4dd1-b3c5-0320c1b487ca None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:01.821629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65d1a-441b-4dd1-b3c5-0320c1b487ca None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:01.821895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a65d1a-441b-4dd1-b3c5-0320c1b487ca None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:01.822541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a65d1a-441b-4dd1-b3c5-0320c1b487ca None admin] RBAC: Adding query filter params (name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:01.822837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a65d1a-441b-4dd1-b3c5-0320c1b487ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:01.824174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a65d1a-441b-4dd1-b3c5-0320c1b487ca None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:01.830215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee2543d-c35f-4662-9f62-e1d169abff4b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:01.836833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 328/646] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:30:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:01.836833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a65d1a-441b-4dd1-b3c5-0320c1b487ca None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:01.871026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad2c8b-dc0f-47a8-87ef-1b4337b6b4d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:01.871682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad2c8b-dc0f-47a8-87ef-1b4337b6b4d3 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:01.871958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad2c8b-dc0f-47a8-87ef-1b4337b6b4d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:01.872230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad2c8b-dc0f-47a8-87ef-1b4337b6b4d3 None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/a3f142a9e141440cbc854ade4f4135a0/roles/29088ad68ec54d74bee2847e2272e4e0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:01.881039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:01.887645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad2c8b-dc0f-47a8-87ef-1b4337b6b4d3 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=a3f142a9e141440cbc854ade4f4135a0, role_id=29088ad68ec54d74bee2847e2272e4e0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:01.889733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ad2c8b-dc0f-47a8-87ef-1b4337b6b4d3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:01.912785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 319/647] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:30:01 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/a3f142a9e141440cbc854ade4f4135a0/roles/29088ad68ec54d74bee2847e2272e4e0 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:30:01.912785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad2c8b-dc0f-47a8-87ef-1b4337b6b4d3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:01.965504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a65d1a-441b-4dd1-b3c5-0320c1b487ca None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:02.005096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb747f7-3260-4569-8790-75393ead70e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:02.006136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb747f7-3260-4569-8790-75393ead70e3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:02.006514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb747f7-3260-4569-8790-75393ead70e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:02.006885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb747f7-3260-4569-8790-75393ead70e3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:02.010049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:02.019082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb747f7-3260-4569-8790-75393ead70e3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:02.021472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb747f7-3260-4569-8790-75393ead70e3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-9bb747f7-3260-4569-8790-75393ead70e3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:02.037234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:02.038198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:02.039160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:02.039160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:02.039160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:02.039160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:02.039160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:02.039160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:02.039160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:02.039160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:02.039160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 22:30:02.039160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:02.041659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 329/648] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:30:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:02.042654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb747f7-3260-4569-8790-75393ead70e3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:02.056718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ad2c8b-dc0f-47a8-87ef-1b4337b6b4d3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:02.102496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac39b4-5457-4335-93cd-d1452446c427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:02.103104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac39b4-5457-4335-93cd-d1452446c427 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:02.103377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac39b4-5457-4335-93cd-d1452446c427 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:02.103647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dac39b4-5457-4335-93cd-d1452446c427 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:02.104314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac39b4-5457-4335-93cd-d1452446c427 None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:02.104608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac39b4-5457-4335-93cd-d1452446c427 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:02.107326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dac39b4-5457-4335-93cd-d1452446c427 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:02.118323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 320/649] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:30:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:02.119208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac39b4-5457-4335-93cd-d1452446c427 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:02.170716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb747f7-3260-4569-8790-75393ead70e3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:02.172507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:02.210651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6e885-c5a2-4e14-ac0c-420ccf0f38c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:02.211274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6e885-c5a2-4e14-ac0c-420ccf0f38c3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:02.211549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6e885-c5a2-4e14-ac0c-420ccf0f38c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:02.211819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f6e885-c5a2-4e14-ac0c-420ccf0f38c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:02.224355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6e885-c5a2-4e14-ac0c-420ccf0f38c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:02.225848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f6e885-c5a2-4e14-ac0c-420ccf0f38c3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-76f6e885-c5a2-4e14-ac0c-420ccf0f38c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:02.242862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:02.243814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:02.244699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:02.244699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:02.244699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:02.244699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 330/650] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:02.245419 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6e885-c5a2-4e14-ac0c-420ccf0f38c3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:02.249971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dac39b4-5457-4335-93cd-d1452446c427 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:02.286936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:02.293188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f440b7a-7f8a-4682-83a0-fb2bc118440a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:02.293937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f440b7a-7f8a-4682-83a0-fb2bc118440a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:02.294328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f440b7a-7f8a-4682-83a0-fb2bc118440a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:02.294749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f440b7a-7f8a-4682-83a0-fb2bc118440a None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:02.295962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f440b7a-7f8a-4682-83a0-fb2bc118440a None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:02.296340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f440b7a-7f8a-4682-83a0-fb2bc118440a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:02.298227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f440b7a-7f8a-4682-83a0-fb2bc118440a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:02.321589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 321/651] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Sep 6 22:30:02 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:02.322612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f440b7a-7f8a-4682-83a0-fb2bc118440a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:02.366669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f6e885-c5a2-4e14-ac0c-420ccf0f38c3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:02.416653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b2f14-8efd-4021-a71c-2453eb2b78b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:02.417766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b2f14-8efd-4021-a71c-2453eb2b78b2 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:02.418391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b2f14-8efd-4021-a71c-2453eb2b78b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:02.418736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123b2f14-8efd-4021-a71c-2453eb2b78b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:02.432915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f440b7a-7f8a-4682-83a0-fb2bc118440a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:02.433470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b2f14-8efd-4021-a71c-2453eb2b78b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:02.436108 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123b2f14-8efd-4021-a71c-2453eb2b78b2 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-123b2f14-8efd-4021-a71c-2453eb2b78b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:02.452883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:02.454090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:02.455203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:02.455203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:02.455203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:02.460255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 331/652] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:02.460255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b2f14-8efd-4021-a71c-2453eb2b78b2 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:02.467553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:02.487193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc38f1e-d279-4ce4-8cdd-b77ea1326ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:02.488660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc38f1e-d279-4ce4-8cdd-b77ea1326ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:02.489756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc38f1e-d279-4ce4-8cdd-b77ea1326ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:02.490332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc38f1e-d279-4ce4-8cdd-b77ea1326ad5 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:02.491243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc38f1e-d279-4ce4-8cdd-b77ea1326ad5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:02.491943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc38f1e-d279-4ce4-8cdd-b77ea1326ad5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:02.494263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc38f1e-d279-4ce4-8cdd-b77ea1326ad5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:02.511342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 322/653] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:02.512205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc38f1e-d279-4ce4-8cdd-b77ea1326ad5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:02.576937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:02.592430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123b2f14-8efd-4021-a71c-2453eb2b78b2 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:02.620695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc38f1e-d279-4ce4-8cdd-b77ea1326ad5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:02.637176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacd0d5-1431-4fa3-b212-d21a0196da05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:02.638170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacd0d5-1431-4fa3-b212-d21a0196da05 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:02.638916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacd0d5-1431-4fa3-b212-d21a0196da05 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:02.639375 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacd0d5-1431-4fa3-b212-d21a0196da05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:02.659255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95fcf4-1fe5-46c7-9821-f7360d02af10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:02.660904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95fcf4-1fe5-46c7-9821-f7360d02af10 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:02.661638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95fcf4-1fe5-46c7-9821-f7360d02af10 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:02.662085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe95fcf4-1fe5-46c7-9821-f7360d02af10 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:02.663556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95fcf4-1fe5-46c7-9821-f7360d02af10 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:02.664034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95fcf4-1fe5-46c7-9821-f7360d02af10 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:02.666101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe95fcf4-1fe5-46c7-9821-f7360d02af10 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:02.687629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 323/654] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:02.687629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95fcf4-1fe5-46c7-9821-f7360d02af10 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:02.719779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacd0d5-1431-4fa3-b212-d21a0196da05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:02.721322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacd0d5-1431-4fa3-b212-d21a0196da05 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:02.725552 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:02.814796 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe95fcf4-1fe5-46c7-9821-f7360d02af10 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:02.827337 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:02.855142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97cb39-b85a-4312-84f6-4b4a53b1aee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:02.855776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97cb39-b85a-4312-84f6-4b4a53b1aee3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:02.856053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97cb39-b85a-4312-84f6-4b4a53b1aee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:02.856324 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97cb39-b85a-4312-84f6-4b4a53b1aee3 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:02.862367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 332/655] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:30:02.864495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacd0d5-1431-4fa3-b212-d21a0196da05 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:02.875452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97cb39-b85a-4312-84f6-4b4a53b1aee3 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:02.877545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97cb39-b85a-4312-84f6-4b4a53b1aee3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-8d97cb39-b85a-4312-84f6-4b4a53b1aee3 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:02.891916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:02.892962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:02.893895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:02.893895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:02.893895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:02.893895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:02.893895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:02.893895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:02.893895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:02.893895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:02.893895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 06 22:30:02.893895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:02.893895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 324/656] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:30:02 2022] GET /identity/v3/users/neutron => generated 85 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:02.894551 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97cb39-b85a-4312-84f6-4b4a53b1aee3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:02.999159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacd0d5-1431-4fa3-b212-d21a0196da05 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:03.003209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:03.021639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97cb39-b85a-4312-84f6-4b4a53b1aee3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:03.046504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b74aac-17d3-4323-8f39-5ad52ca1a73b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:03.048004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b74aac-17d3-4323-8f39-5ad52ca1a73b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:03.048594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b74aac-17d3-4323-8f39-5ad52ca1a73b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:03.049078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b74aac-17d3-4323-8f39-5ad52ca1a73b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:03.058905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bd5e4-14db-463d-8513-7f07f7916afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:03.059508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b74aac-17d3-4323-8f39-5ad52ca1a73b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:03.059618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bd5e4-14db-463d-8513-7f07f7916afc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:03.059783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bd5e4-14db-463d-8513-7f07f7916afc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:03.060053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447bd5e4-14db-463d-8513-7f07f7916afc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:03.061641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b74aac-17d3-4323-8f39-5ad52ca1a73b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:03.072295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bd5e4-14db-463d-8513-7f07f7916afc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:03.073789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447bd5e4-14db-463d-8513-7f07f7916afc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-52b74aac-17d3-4323-8f39-5ad52ca1a73b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:03.078248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:03.079153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:03.080088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:03.080088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:03.080088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:03.080088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:03.080088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:03.080088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:03.080088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:03.080088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:03.080088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 06 22:30:03.080088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:03.087231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 333/657] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:30:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:03.087231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b74aac-17d3-4323-8f39-5ad52ca1a73b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-447bd5e4-14db-463d-8513-7f07f7916afc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:03.095844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:03.096811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:03.098420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:03.098420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:03.098420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:03.102333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 325/658] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:03.103198 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bd5e4-14db-463d-8513-7f07f7916afc None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:03.130235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:03.224727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b74aac-17d3-4323-8f39-5ad52ca1a73b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:03.225891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447bd5e4-14db-463d-8513-7f07f7916afc None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:03.266666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b788a-6853-450d-8eaa-40b611d2978e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:03.267265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b788a-6853-450d-8eaa-40b611d2978e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:03.267623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b788a-6853-450d-8eaa-40b611d2978e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:03.267888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-149b788a-6853-450d-8eaa-40b611d2978e None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:03.268609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b788a-6853-450d-8eaa-40b611d2978e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:03.268938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b788a-6853-450d-8eaa-40b611d2978e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:03.270927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-149b788a-6853-450d-8eaa-40b611d2978e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:03.282453 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fd13a-67a4-4af6-a708-2a78dc31db24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:03.283380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd13a-67a4-4af6-a708-2a78dc31db24 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:03.283856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd13a-67a4-4af6-a708-2a78dc31db24 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:03.284326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014fd13a-67a4-4af6-a708-2a78dc31db24 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:03.286940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fd13a-67a4-4af6-a708-2a78dc31db24 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:03.287478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fd13a-67a4-4af6-a708-2a78dc31db24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:03.289452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014fd13a-67a4-4af6-a708-2a78dc31db24 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:03.295274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 334/659] 10.222.0.39 () {56 vars in 1139 bytes} [Tue Sep 6 22:30:03 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:03.296191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b788a-6853-450d-8eaa-40b611d2978e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:03.315667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 326/660] 10.222.0.39 () {56 vars in 1137 bytes} [Tue Sep 6 22:30:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:03.318155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fd13a-67a4-4af6-a708-2a78dc31db24 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:03.323945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:03.415450 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-149b788a-6853-450d-8eaa-40b611d2978e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:03.435574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:03.460655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9079f0-27f6-4ec1-a307-cc91d6db57c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:03.461052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014fd13a-67a4-4af6-a708-2a78dc31db24 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:03.461287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9079f0-27f6-4ec1-a307-cc91d6db57c3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:03.461562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9079f0-27f6-4ec1-a307-cc91d6db57c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:03.461824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9079f0-27f6-4ec1-a307-cc91d6db57c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:03.462451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9079f0-27f6-4ec1-a307-cc91d6db57c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:03.462748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9079f0-27f6-4ec1-a307-cc91d6db57c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:03.464198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9079f0-27f6-4ec1-a307-cc91d6db57c3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:03.477027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 335/661] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:03.477027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9079f0-27f6-4ec1-a307-cc91d6db57c3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:03.498187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42313a9e-8067-4ba6-8697-baeb7fc6fcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:03.498793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313a9e-8067-4ba6-8697-baeb7fc6fcfb None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:03.499073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313a9e-8067-4ba6-8697-baeb7fc6fcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:03.499354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42313a9e-8067-4ba6-8697-baeb7fc6fcfb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:03.574124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:03.576597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42313a9e-8067-4ba6-8697-baeb7fc6fcfb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:03.578110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42313a9e-8067-4ba6-8697-baeb7fc6fcfb None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:03.596182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9079f0-27f6-4ec1-a307-cc91d6db57c3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:03.634407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae8f40-54c4-40a0-ad6a-81ab27f9160d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:03.635007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae8f40-54c4-40a0-ad6a-81ab27f9160d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:03.635287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae8f40-54c4-40a0-ad6a-81ab27f9160d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:03.635660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ae8f40-54c4-40a0-ad6a-81ab27f9160d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:03.666845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:03.688895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 327/662] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:30:03.690428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662369f-a748-4495-92df-ba2f36dfd5ce None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:03.690900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662369f-a748-4495-92df-ba2f36dfd5ce None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:03.691590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662369f-a748-4495-92df-ba2f36dfd5ce None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:03.693439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 328/663] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:03.694357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662369f-a748-4495-92df-ba2f36dfd5ce None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:03.715804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae8f40-54c4-40a0-ad6a-81ab27f9160d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:03.717283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ae8f40-54c4-40a0-ad6a-81ab27f9160d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:03.802020 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662369f-a748-4495-92df-ba2f36dfd5ce None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:03.804215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:03.829697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 336/664] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:30:03.829697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae8f40-54c4-40a0-ad6a-81ab27f9160d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:03.844215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb30ba2-392a-4781-a9de-2957a11bb251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:03.845210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb30ba2-392a-4781-a9de-2957a11bb251 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:03.845671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb30ba2-392a-4781-a9de-2957a11bb251 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:03.846100 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb30ba2-392a-4781-a9de-2957a11bb251 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:03.853987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb30ba2-392a-4781-a9de-2957a11bb251 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:03.855599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb30ba2-392a-4781-a9de-2957a11bb251 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-8eb30ba2-392a-4781-a9de-2957a11bb251 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:03.868656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:03.869773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:03.870856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:03.870856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:03.870856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:03.870856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:03.870856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:03.870856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:03.870856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:03.870856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:03.872702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 329/665] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:03.873866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85737cb4-dad7-45b3-81fa-016a9430dcb8 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:03.874144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85737cb4-dad7-45b3-81fa-016a9430dcb8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:03.874412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85737cb4-dad7-45b3-81fa-016a9430dcb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:03.893879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-85737cb4-dad7-45b3-81fa-016a9430dcb8 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:03.895217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-85737cb4-dad7-45b3-81fa-016a9430dcb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:03.932477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:03.945912 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ae8f40-54c4-40a0-ad6a-81ab27f9160d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:03.968154 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 330/666] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:03.969125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85737cb4-dad7-45b3-81fa-016a9430dcb8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:03.995750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c3721-d9b5-466c-93f1-397905d7ed9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:03.996981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c3721-d9b5-466c-93f1-397905d7ed9b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:03.997711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c3721-d9b5-466c-93f1-397905d7ed9b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:03.998609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0c3721-d9b5-466c-93f1-397905d7ed9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.013059 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c3721-d9b5-466c-93f1-397905d7ed9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:04.015636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0c3721-d9b5-466c-93f1-397905d7ed9b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-4a0c3721-d9b5-466c-93f1-397905d7ed9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:04.036913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:04.037894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:04.038811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:04.038811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:04.038811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:04.045283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 337/667] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:04.046363 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c3721-d9b5-466c-93f1-397905d7ed9b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:04.090334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:04.100146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85737cb4-dad7-45b3-81fa-016a9430dcb8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:04.145489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793fb14-c24f-4970-88a0-6b9033e0dde2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:04.146432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793fb14-c24f-4970-88a0-6b9033e0dde2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:04.146869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793fb14-c24f-4970-88a0-6b9033e0dde2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:04.147280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4793fb14-c24f-4970-88a0-6b9033e0dde2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.162113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793fb14-c24f-4970-88a0-6b9033e0dde2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:04.163934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4793fb14-c24f-4970-88a0-6b9033e0dde2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:04.180923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0c3721-d9b5-466c-93f1-397905d7ed9b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-4793fb14-c24f-4970-88a0-6b9033e0dde2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:04.181677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:04.182848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:04.183993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:04.183993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:04.183993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:04.189610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 331/668] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:04.189610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793fb14-c24f-4970-88a0-6b9033e0dde2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:04.217158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:04.234951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b988b-da1e-4a67-a1bb-c8d8fee5561e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:04.236434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b988b-da1e-4a67-a1bb-c8d8fee5561e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:04.236950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b988b-da1e-4a67-a1bb-c8d8fee5561e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:04.237447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b988b-da1e-4a67-a1bb-c8d8fee5561e None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.239266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b988b-da1e-4a67-a1bb-c8d8fee5561e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:04.239608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b988b-da1e-4a67-a1bb-c8d8fee5561e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:04.241408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b988b-da1e-4a67-a1bb-c8d8fee5561e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:04.253459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 338/669] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:04.255031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b988b-da1e-4a67-a1bb-c8d8fee5561e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:04.296810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4793fb14-c24f-4970-88a0-6b9033e0dde2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:04.330279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc72713e-efe9-4cf0-a958-626996762f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:04.330876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72713e-efe9-4cf0-a958-626996762f4c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:04.331151 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72713e-efe9-4cf0-a958-626996762f4c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:04.331421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc72713e-efe9-4cf0-a958-626996762f4c None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.332078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72713e-efe9-4cf0-a958-626996762f4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:04.332380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72713e-efe9-4cf0-a958-626996762f4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:04.333766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc72713e-efe9-4cf0-a958-626996762f4c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:04.345075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 332/670] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:04.346213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74eea6-179a-4dc0-b2f8-b1668178e74a None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:04.346492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74eea6-179a-4dc0-b2f8-b1668178e74a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:04.346765 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f74eea6-179a-4dc0-b2f8-b1668178e74a None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.347162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:04.352341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 333/671] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:04.352341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74eea6-179a-4dc0-b2f8-b1668178e74a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:04.358596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b988b-da1e-4a67-a1bb-c8d8fee5561e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:04.393687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a2fe4-134a-4638-a3a8-80e48b0fe33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:04.394311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2fe4-134a-4638-a3a8-80e48b0fe33b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:04.394586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2fe4-134a-4638-a3a8-80e48b0fe33b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:04.394855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2fe4-134a-4638-a3a8-80e48b0fe33b None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.395484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a2fe4-134a-4638-a3a8-80e48b0fe33b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:04.395782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a2fe4-134a-4638-a3a8-80e48b0fe33b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:04.397169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a2fe4-134a-4638-a3a8-80e48b0fe33b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:04.407977 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 339/672] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:04.408737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a2fe4-134a-4638-a3a8-80e48b0fe33b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:04.433203 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:04.452673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f74eea6-179a-4dc0-b2f8-b1668178e74a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:04.485747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea85af8-1235-4a1d-89a2-f1cabbe5bd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:04.486363 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea85af8-1235-4a1d-89a2-f1cabbe5bd35 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:04.486643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea85af8-1235-4a1d-89a2-f1cabbe5bd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:04.486902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea85af8-1235-4a1d-89a2-f1cabbe5bd35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.497164 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea85af8-1235-4a1d-89a2-f1cabbe5bd35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-4ea85af8-1235-4a1d-89a2-f1cabbe5bd35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:04.501899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:04.502949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 334/673] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:30:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:04.504404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea85af8-1235-4a1d-89a2-f1cabbe5bd35 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:04.513080 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a2fe4-134a-4638-a3a8-80e48b0fe33b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:04.547119 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad80ef95-986b-4e9f-b283-e3d8e660f9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:04.547725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80ef95-986b-4e9f-b283-e3d8e660f9f9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:04.547999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80ef95-986b-4e9f-b283-e3d8e660f9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:04.548268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad80ef95-986b-4e9f-b283-e3d8e660f9f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.549017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80ef95-986b-4e9f-b283-e3d8e660f9f9 None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=d2da9e23277a4de2a61b9727737ab8f5) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:04.549331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80ef95-986b-4e9f-b283-e3d8e660f9f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:04.551332 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad80ef95-986b-4e9f-b283-e3d8e660f9f9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:04.564987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 340/674] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:04 2022] GET /identity/v3/role_assignments?user.id=d2da9e23277a4de2a61b9727737ab8f5&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:04.564987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a138c8-e108-4e5d-9cc8-d842cb5a6f16 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:04.564987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a138c8-e108-4e5d-9cc8-d842cb5a6f16 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:04.564987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a138c8-e108-4e5d-9cc8-d842cb5a6f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.581229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-54a138c8-e108-4e5d-9cc8-d842cb5a6f16 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:04.583229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-54a138c8-e108-4e5d-9cc8-d842cb5a6f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:04.598960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:04.633311 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea85af8-1235-4a1d-89a2-f1cabbe5bd35 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:04.667446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 341/675] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:04.668551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a138c8-e108-4e5d-9cc8-d842cb5a6f16 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:04.684782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f867e-2ce3-4266-b4b7-4925bc9864c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:04.685629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f867e-2ce3-4266-b4b7-4925bc9864c1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:04.686034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f867e-2ce3-4266-b4b7-4925bc9864c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:04.686412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4f867e-2ce3-4266-b4b7-4925bc9864c1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.695588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f867e-2ce3-4266-b4b7-4925bc9864c1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:04.697445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4f867e-2ce3-4266-b4b7-4925bc9864c1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-8d4f867e-2ce3-4266-b4b7-4925bc9864c1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:04.711783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:04.712807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:04.713740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:04.713740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:04.713740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:04.713740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:04.713740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:04.713740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:04.713740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:04.713740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:04.718413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 335/676] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:04.719448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f867e-2ce3-4266-b4b7-4925bc9864c1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:04.722824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:04.790918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a138c8-e108-4e5d-9cc8-d842cb5a6f16 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:04.836337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20c545-ba1e-46af-a572-70aa94195340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:04.837470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c545-ba1e-46af-a572-70aa94195340 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:04.837761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c545-ba1e-46af-a572-70aa94195340 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:04.838036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20c545-ba1e-46af-a572-70aa94195340 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.839552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4f867e-2ce3-4266-b4b7-4925bc9864c1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:04.848156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20c545-ba1e-46af-a572-70aa94195340 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:04.848156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20c545-ba1e-46af-a572-70aa94195340 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-2f20c545-ba1e-46af-a572-70aa94195340 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:04.865921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:04.867019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:04.867908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:04.867908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:04.867908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:04.867908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:04.867908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:04.867908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:04.867908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:04.867908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:04.870756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 342/677] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:04.871975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20c545-ba1e-46af-a572-70aa94195340 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:04.889708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74129f-33c5-4136-b44d-a0b65a91cc56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:04.889708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74129f-33c5-4136-b44d-a0b65a91cc56 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:04.889708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74129f-33c5-4136-b44d-a0b65a91cc56 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:04.889708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd74129f-33c5-4136-b44d-a0b65a91cc56 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:04.890323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74129f-33c5-4136-b44d-a0b65a91cc56 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:04.890323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74129f-33c5-4136-b44d-a0b65a91cc56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:04.890323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd74129f-33c5-4136-b44d-a0b65a91cc56 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:04.897167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:04.898588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 336/678] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:30:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:04.899493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74129f-33c5-4136-b44d-a0b65a91cc56 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:04.985808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:04.994654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20c545-ba1e-46af-a572-70aa94195340 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:05.032830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd74129f-33c5-4136-b44d-a0b65a91cc56 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:05.037203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450631e-11b8-4844-87b1-3d78aec169ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:05.037344 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450631e-11b8-4844-87b1-3d78aec169ae None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:05.037514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450631e-11b8-4844-87b1-3d78aec169ae None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:05.037774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450631e-11b8-4844-87b1-3d78aec169ae None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:05.038460 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450631e-11b8-4844-87b1-3d78aec169ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:05.038757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450631e-11b8-4844-87b1-3d78aec169ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:05.040175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450631e-11b8-4844-87b1-3d78aec169ae None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:05.052309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 343/679] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:05.053239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450631e-11b8-4844-87b1-3d78aec169ae None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:05.075972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d4838-360c-448d-ba6d-16d8852e7fb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:05.076575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d4838-360c-448d-ba6d-16d8852e7fb9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:05.076884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d4838-360c-448d-ba6d-16d8852e7fb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:05.077155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835d4838-360c-448d-ba6d-16d8852e7fb9 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:05.077840 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d4838-360c-448d-ba6d-16d8852e7fb9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:05.078139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d4838-360c-448d-ba6d-16d8852e7fb9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:05.079990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835d4838-360c-448d-ba6d-16d8852e7fb9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:05.093118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 337/680] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:05.099645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d4838-360c-448d-ba6d-16d8852e7fb9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:05.126512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:05.164472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450631e-11b8-4844-87b1-3d78aec169ae None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:05.204121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e25a6-60d1-4c5b-bd8f-3e9f03219f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:05.204769 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e25a6-60d1-4c5b-bd8f-3e9f03219f6e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:05.205067 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e25a6-60d1-4c5b-bd8f-3e9f03219f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:05.205347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1e25a6-60d1-4c5b-bd8f-3e9f03219f6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:05.207719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835d4838-360c-448d-ba6d-16d8852e7fb9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:05.216517 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e25a6-60d1-4c5b-bd8f-3e9f03219f6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:05.217998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1e25a6-60d1-4c5b-bd8f-3e9f03219f6e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-ca1e25a6-60d1-4c5b-bd8f-3e9f03219f6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:05.232810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:05.233917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:05.234948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:05.234948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:05.234948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:05.236424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:05.239952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 344/681] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:05.241314 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e25a6-60d1-4c5b-bd8f-3e9f03219f6e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:05.250273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eda60c-4957-4d5e-880c-acce35d29bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:05.250907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eda60c-4957-4d5e-880c-acce35d29bcd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:05.251185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eda60c-4957-4d5e-880c-acce35d29bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:05.251456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eda60c-4957-4d5e-880c-acce35d29bcd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:05.262031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eda60c-4957-4d5e-880c-acce35d29bcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e8eda60c-4957-4d5e-880c-acce35d29bcd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:05.266830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:05.267725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:05.268579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 338/682] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:30:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:05.269304 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eda60c-4957-4d5e-880c-acce35d29bcd None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:05.351984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1e25a6-60d1-4c5b-bd8f-3e9f03219f6e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:05.373085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:05.384877 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eda60c-4957-4d5e-880c-acce35d29bcd None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:05.389191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324b5c8-8fc6-499a-88c1-202b424a90c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:05.389798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324b5c8-8fc6-499a-88c1-202b424a90c1 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:05.390076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324b5c8-8fc6-499a-88c1-202b424a90c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:05.390347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7324b5c8-8fc6-499a-88c1-202b424a90c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:05.391071 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324b5c8-8fc6-499a-88c1-202b424a90c1 None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=2658435f74a947f39e207465ebff22d8) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:05.391379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324b5c8-8fc6-499a-88c1-202b424a90c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:05.393037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7324b5c8-8fc6-499a-88c1-202b424a90c1 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:05.399679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 345/683] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:05 2022] GET /identity/v3/role_assignments?user.id=2658435f74a947f39e207465ebff22d8&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:05.400470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324b5c8-8fc6-499a-88c1-202b424a90c1 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:05.440970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1692b-2a9f-41b7-ae7f-94964ede5efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:05.442500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1692b-2a9f-41b7-ae7f-94964ede5efe None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:05.443917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1692b-2a9f-41b7-ae7f-94964ede5efe None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:05.444457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1692b-2a9f-41b7-ae7f-94964ede5efe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:05.445446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1692b-2a9f-41b7-ae7f-94964ede5efe None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=42cf265450454ff9a27a29093336205e) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:05.445969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1692b-2a9f-41b7-ae7f-94964ede5efe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:05.447658 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a1692b-2a9f-41b7-ae7f-94964ede5efe None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:05.458403 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 339/684] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:05 2022] GET /identity/v3/role_assignments?user.id=42cf265450454ff9a27a29093336205e&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:05.459763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1692b-2a9f-41b7-ae7f-94964ede5efe None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:05.530937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:05.602029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a1692b-2a9f-41b7-ae7f-94964ede5efe None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:05.615969 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7324b5c8-8fc6-499a-88c1-202b424a90c1 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:05.653131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07738390-cb68-4cf0-bdc8-1813d7dd4f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:05.654231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07738390-cb68-4cf0-bdc8-1813d7dd4f73 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:05.655015 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07738390-cb68-4cf0-bdc8-1813d7dd4f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:05.655707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07738390-cb68-4cf0-bdc8-1813d7dd4f73 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:05.656949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07738390-cb68-4cf0-bdc8-1813d7dd4f73 None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:05.657784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07738390-cb68-4cf0-bdc8-1813d7dd4f73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:05.659879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07738390-cb68-4cf0-bdc8-1813d7dd4f73 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:05.666210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6e864-0adf-4a39-a400-d6fd818d86ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:05.666825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6e864-0adf-4a39-a400-d6fd818d86ce None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:05.667094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6e864-0adf-4a39-a400-d6fd818d86ce None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:05.667357 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f6e864-0adf-4a39-a400-d6fd818d86ce None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:05.668000 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6e864-0adf-4a39-a400-d6fd818d86ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:05.668298 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6e864-0adf-4a39-a400-d6fd818d86ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:05.669797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f6e864-0adf-4a39-a400-d6fd818d86ce None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:05.674378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 340/685] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:30:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:05.680918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07738390-cb68-4cf0-bdc8-1813d7dd4f73 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:05.687628 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 346/686] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:05.694136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6e864-0adf-4a39-a400-d6fd818d86ce None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:05.788604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:05.883112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f6e864-0adf-4a39-a400-d6fd818d86ce None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:05.928791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b02b1-2c75-4ecc-927c-6459c6ff051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:05.929154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b02b1-2c75-4ecc-927c-6459c6ff051f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:05.932800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b02b1-2c75-4ecc-927c-6459c6ff051f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:05.932800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158b02b1-2c75-4ecc-927c-6459c6ff051f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:05.941765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b02b1-2c75-4ecc-927c-6459c6ff051f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:05.941765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158b02b1-2c75-4ecc-927c-6459c6ff051f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:05.957134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07738390-cb68-4cf0-bdc8-1813d7dd4f73 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-158b02b1-2c75-4ecc-927c-6459c6ff051f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:05.957635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:05.958510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:05.959459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:05.959459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:05.959459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:05.959459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:05.959459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:05.959459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:05.959459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:05.959459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:05.959459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:30:05.959459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:05.959459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 347/687] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Sep 6 22:30:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:05.968795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b02b1-2c75-4ecc-927c-6459c6ff051f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:05.982965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:06.040845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c91842-814c-4b04-9ebd-a7dd58e08736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:06.044782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c91842-814c-4b04-9ebd-a7dd58e08736 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:06.044782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c91842-814c-4b04-9ebd-a7dd58e08736 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:06.044782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c91842-814c-4b04-9ebd-a7dd58e08736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:06.080794 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c91842-814c-4b04-9ebd-a7dd58e08736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:06.084520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c91842-814c-4b04-9ebd-a7dd58e08736 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-94c91842-814c-4b04-9ebd-a7dd58e08736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:06.130563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:06.131579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:06.132743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:06.132743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:06.132743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:06.132743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 341/688] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 452 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:06.140844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c91842-814c-4b04-9ebd-a7dd58e08736 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:06.230805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158b02b1-2c75-4ecc-927c-6459c6ff051f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:06.238633 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:06.316123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63d379-dbe0-40bc-af12-9a19ec430ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:06.317138 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63d379-dbe0-40bc-af12-9a19ec430ace None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:06.317655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63d379-dbe0-40bc-af12-9a19ec430ace None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:06.318129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db63d379-dbe0-40bc-af12-9a19ec430ace None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:06.319070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63d379-dbe0-40bc-af12-9a19ec430ace None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:06.328778 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63d379-dbe0-40bc-af12-9a19ec430ace None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:06.331582 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db63d379-dbe0-40bc-af12-9a19ec430ace None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:06.383609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 348/689] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Sep 6 22:30:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:06.396733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63d379-dbe0-40bc-af12-9a19ec430ace None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:06.432421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c91842-814c-4b04-9ebd-a7dd58e08736 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:06.558832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66706fe7-ce32-4d62-86fd-8b6ab611e066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:06.559452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66706fe7-ce32-4d62-86fd-8b6ab611e066 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:06.559730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66706fe7-ce32-4d62-86fd-8b6ab611e066 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:06.559997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66706fe7-ce32-4d62-86fd-8b6ab611e066 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:06.560644 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66706fe7-ce32-4d62-86fd-8b6ab611e066 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:06.560961 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66706fe7-ce32-4d62-86fd-8b6ab611e066 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:06.562321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66706fe7-ce32-4d62-86fd-8b6ab611e066 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:06.596482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 342/690] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 457 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:06.602465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66706fe7-ce32-4d62-86fd-8b6ab611e066 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:06.633170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:06.684546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db63d379-dbe0-40bc-af12-9a19ec430ace None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:06.746403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb14ae-3570-4c71-b1f6-55cf9c555869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:06.747298 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb14ae-3570-4c71-b1f6-55cf9c555869 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:06.748563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb14ae-3570-4c71-b1f6-55cf9c555869 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:06.749232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccb14ae-3570-4c71-b1f6-55cf9c555869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:06.786347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66706fe7-ce32-4d62-86fd-8b6ab611e066 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:06.813578 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:06.871712 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb14ae-3570-4c71-b1f6-55cf9c555869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:06.873603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccb14ae-3570-4c71-b1f6-55cf9c555869 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:06.877574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ea941-c685-461c-af93-7e62a607fe38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:06.878237 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ea941-c685-461c-af93-7e62a607fe38 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:06.878522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ea941-c685-461c-af93-7e62a607fe38 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:06.879373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230ea941-c685-461c-af93-7e62a607fe38 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:06.905563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ea941-c685-461c-af93-7e62a607fe38 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:06.907020 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230ea941-c685-461c-af93-7e62a607fe38 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-230ea941-c685-461c-af93-7e62a607fe38 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:06.942619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:06.943629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:06.945218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:06.945218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:06.945218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:06.945218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:06.945218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:06.945218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:06.945218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:06.945218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:06.945218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:30:06.945218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:06.945218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 343/691] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:30:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:06.949506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ea941-c685-461c-af93-7e62a607fe38 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:06.991649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 349/692] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 595 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:30:06.997273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb14ae-3570-4c71-b1f6-55cf9c555869 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:07.099782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230ea941-c685-461c-af93-7e62a607fe38 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:07.100459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:07.113454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:07.149651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f4dd1-d031-4f16-a810-65d0517b1670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:07.150193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f4dd1-d031-4f16-a810-65d0517b1670 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:07.150481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f4dd1-d031-4f16-a810-65d0517b1670 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:07.150758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f4dd1-d031-4f16-a810-65d0517b1670 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:07.151498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f4dd1-d031-4f16-a810-65d0517b1670 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:07.151806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f4dd1-d031-4f16-a810-65d0517b1670 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:07.155814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccb14ae-3570-4c71-b1f6-55cf9c555869 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:07.157816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f4dd1-d031-4f16-a810-65d0517b1670 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:07.188740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 344/693] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Sep 6 22:30:06 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:07.196663 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f4dd1-d031-4f16-a810-65d0517b1670 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:07.223500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488113b4-a930-4ec5-8cb8-cd917c2ec8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:07.224161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488113b4-a930-4ec5-8cb8-cd917c2ec8e5 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:07.224441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488113b4-a930-4ec5-8cb8-cd917c2ec8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:07.224715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488113b4-a930-4ec5-8cb8-cd917c2ec8e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:07.240454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488113b4-a930-4ec5-8cb8-cd917c2ec8e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:07.241905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488113b4-a930-4ec5-8cb8-cd917c2ec8e5 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-488113b4-a930-4ec5-8cb8-cd917c2ec8e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:07.269593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:07.270614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:07.272103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:07.272103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:07.272103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:07.272103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 350/694] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:07.276323 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488113b4-a930-4ec5-8cb8-cd917c2ec8e5 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:07.331282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f4dd1-d031-4f16-a810-65d0517b1670 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:07.370797 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73d64c5-8601-438e-b333-35751d117c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:07.371775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d64c5-8601-438e-b333-35751d117c41 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:07.372216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d64c5-8601-438e-b333-35751d117c41 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:07.372635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73d64c5-8601-438e-b333-35751d117c41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:07.384989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:07.416504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488113b4-a930-4ec5-8cb8-cd917c2ec8e5 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:07.435625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:07.450436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d64c5-8601-438e-b333-35751d117c41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:07.452382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73d64c5-8601-438e-b333-35751d117c41 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:07.464949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7802925e-eed3-41bf-a199-844d2594b80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:07.465664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802925e-eed3-41bf-a199-844d2594b80f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:07.465951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802925e-eed3-41bf-a199-844d2594b80f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:07.466227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7802925e-eed3-41bf-a199-844d2594b80f None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:07.466870 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802925e-eed3-41bf-a199-844d2594b80f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:07.467173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802925e-eed3-41bf-a199-844d2594b80f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:07.468623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7802925e-eed3-41bf-a199-844d2594b80f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:07.482724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 351/695] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:07.484022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a632-1990-4648-91b2-04b0dcd99ed2 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:07.484310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a632-1990-4648-91b2-04b0dcd99ed2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:07.484586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c7a632-1990-4648-91b2-04b0dcd99ed2 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:07.486210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 352/696] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:07.490618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7a632-1990-4648-91b2-04b0dcd99ed2 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:07.591548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 345/697] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 06 22:30:07.591548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a476fe-bd4b-48c0-a031-b1f771d3e76e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:07.591548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a476fe-bd4b-48c0-a031-b1f771d3e76e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:07.591548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a476fe-bd4b-48c0-a031-b1f771d3e76e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:07.609659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-44a476fe-bd4b-48c0-a031-b1f771d3e76e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:07.610963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-44a476fe-bd4b-48c0-a031-b1f771d3e76e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:07.618694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c7a632-1990-4648-91b2-04b0dcd99ed2 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:07.650427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:07.657318 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b068f2-3275-497f-ac7d-30a078998e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:07.658517 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b068f2-3275-497f-ac7d-30a078998e59 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:07.659018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b068f2-3275-497f-ac7d-30a078998e59 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:07.659538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b068f2-3275-497f-ac7d-30a078998e59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:07.667878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b068f2-3275-497f-ac7d-30a078998e59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:07.669573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b068f2-3275-497f-ac7d-30a078998e59 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-03b068f2-3275-497f-ac7d-30a078998e59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:07.681694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:07.682632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:07.683550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:07.683550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:07.683550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:07.683550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:07.683550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:07.683550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:07.683550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:07.683550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:07.685947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 353/698] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:07.686994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b068f2-3275-497f-ac7d-30a078998e59 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:07.692604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 346/699] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:07.693611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a476fe-bd4b-48c0-a031-b1f771d3e76e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:07.724737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:07.797362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b068f2-3275-497f-ac7d-30a078998e59 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:07.803492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a476fe-bd4b-48c0-a031-b1f771d3e76e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:07.836045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7553a-36d2-45c9-9c67-bff07c1c02fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:07.837280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7553a-36d2-45c9-9c67-bff07c1c02fa None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:07.837788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7553a-36d2-45c9-9c67-bff07c1c02fa None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:07.838253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d7553a-36d2-45c9-9c67-bff07c1c02fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:07.840487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcbe04-6084-4d45-8904-495892ed8424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:07.841134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcbe04-6084-4d45-8904-495892ed8424 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:07.841412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcbe04-6084-4d45-8904-495892ed8424 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:07.841683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfcbe04-6084-4d45-8904-495892ed8424 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:07.842380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcbe04-6084-4d45-8904-495892ed8424 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:07.842681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcbe04-6084-4d45-8904-495892ed8424 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:07.844236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfcbe04-6084-4d45-8904-495892ed8424 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:07.852675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7553a-36d2-45c9-9c67-bff07c1c02fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:07.855119 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d7553a-36d2-45c9-9c67-bff07c1c02fa None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:07.861860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 347/700] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:07.863172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcbe04-6084-4d45-8904-495892ed8424 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-b4d7553a-36d2-45c9-9c67-bff07c1c02fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:07.875294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:07.876239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:07.877940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:07.877940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:07.877940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:07.877940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 354/701] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:07.884319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7553a-36d2-45c9-9c67-bff07c1c02fa None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:07.912821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:07.989002 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfcbe04-6084-4d45-8904-495892ed8424 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:07.995222 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:08.016563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d7553a-36d2-45c9-9c67-bff07c1c02fa None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:08.026227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6be2d-8292-47e6-b37d-816995147166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:08.026639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6be2d-8292-47e6-b37d-816995147166 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.026918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6be2d-8292-47e6-b37d-816995147166 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.027189 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac6be2d-8292-47e6-b37d-816995147166 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.027916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6be2d-8292-47e6-b37d-816995147166 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=b16947313dad4d59a97cc8b016ee54c1) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:08.028222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6be2d-8292-47e6-b37d-816995147166 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:08.029688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac6be2d-8292-47e6-b37d-816995147166 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:08.040904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 348/702] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:07 2022] GET /identity/v3/role_assignments?user.id=b16947313dad4d59a97cc8b016ee54c1&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:08.059146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726675f-95bc-4132-b606-cb6c6a49c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:08.060129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726675f-95bc-4132-b606-cb6c6a49c7bf None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.060619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726675f-95bc-4132-b606-cb6c6a49c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.061136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f726675f-95bc-4132-b606-cb6c6a49c7bf None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.062023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726675f-95bc-4132-b606-cb6c6a49c7bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:08.062526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726675f-95bc-4132-b606-cb6c6a49c7bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:08.064455 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f726675f-95bc-4132-b606-cb6c6a49c7bf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:08.079918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 355/703] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:08.086334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6be2d-8292-47e6-b37d-816995147166 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:08.088327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726675f-95bc-4132-b606-cb6c6a49c7bf None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:08.226616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f726675f-95bc-4132-b606-cb6c6a49c7bf None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:08.255096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:08.270820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac6be2d-8292-47e6-b37d-816995147166 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:08.274507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d3077f-4139-466f-b1a2-ef1f3a473d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:08.275592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3077f-4139-466f-b1a2-ef1f3a473d9c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.276070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3077f-4139-466f-b1a2-ef1f3a473d9c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.276539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d3077f-4139-466f-b1a2-ef1f3a473d9c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.277394 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3077f-4139-466f-b1a2-ef1f3a473d9c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:08.279050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d3077f-4139-466f-b1a2-ef1f3a473d9c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-a8d3077f-4139-466f-b1a2-ef1f3a473d9c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:08.293616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:08.294905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 22:30:08.295746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:08.297930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 356/704] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Sep 6 22:30:08 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:08.298997 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ae566-56de-446c-bce6-a2746cd1dbf4 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.299272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ae566-56de-446c-bce6-a2746cd1dbf4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.299539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641ae566-56de-446c-bce6-a2746cd1dbf4 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.301327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 357/705] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:08.310157 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ae566-56de-446c-bce6-a2746cd1dbf4 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:08.321890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d403dc70-ec97-4b64-ae88-b60f26440541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:08.322789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403dc70-ec97-4b64-ae88-b60f26440541 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.323249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403dc70-ec97-4b64-ae88-b60f26440541 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.323727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d403dc70-ec97-4b64-ae88-b60f26440541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.326369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:08.333725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403dc70-ec97-4b64-ae88-b60f26440541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:08.335474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d403dc70-ec97-4b64-ae88-b60f26440541 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-d403dc70-ec97-4b64-ae88-b60f26440541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:08.348389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:08.349484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:08.350648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:08.350648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:08.350648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:08.350648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:08.350648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:08.350648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:08.350648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:08.350648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:08.352896 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 349/706] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:08.354335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d3e1e-e07f-4d00-9b28-a311daa79a42 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.354885 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d3e1e-e07f-4d00-9b28-a311daa79a42 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.355362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64d3e1e-e07f-4d00-9b28-a311daa79a42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.375447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b64d3e1e-e07f-4d00-9b28-a311daa79a42 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:08.377114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b64d3e1e-e07f-4d00-9b28-a311daa79a42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:08.429456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641ae566-56de-446c-bce6-a2746cd1dbf4 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:08.455689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 350/707] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:08.456721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d3e1e-e07f-4d00-9b28-a311daa79a42 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:08.474311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ac5d30-2ff4-41d0-b963-d7ec85d5dd29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:08.475331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac5d30-2ff4-41d0-b963-d7ec85d5dd29 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.476108 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac5d30-2ff4-41d0-b963-d7ec85d5dd29 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.476575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ac5d30-2ff4-41d0-b963-d7ec85d5dd29 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.477417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac5d30-2ff4-41d0-b963-d7ec85d5dd29 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:08.477899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac5d30-2ff4-41d0-b963-d7ec85d5dd29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:08.479910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ac5d30-2ff4-41d0-b963-d7ec85d5dd29 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:08.485823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 358/708] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Sep 6 22:30:08 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:08.487809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14536af7-dc6f-4cf4-a724-41000cca3cab None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.488340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14536af7-dc6f-4cf4-a724-41000cca3cab None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.488948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14536af7-dc6f-4cf4-a724-41000cca3cab None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.495124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 359/709] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:08.495124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14536af7-dc6f-4cf4-a724-41000cca3cab None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:08.559639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:08.582371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64d3e1e-e07f-4d00-9b28-a311daa79a42 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:08.598740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:08.622581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a21ed6-a13c-45f9-9221-4a52a5b8e944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:08.623678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21ed6-a13c-45f9-9221-4a52a5b8e944 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.624292 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21ed6-a13c-45f9-9221-4a52a5b8e944 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.624790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a21ed6-a13c-45f9-9221-4a52a5b8e944 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.625728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a21ed6-a13c-45f9-9221-4a52a5b8e944 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:08.626347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a21ed6-a13c-45f9-9221-4a52a5b8e944 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:08.628142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a21ed6-a13c-45f9-9221-4a52a5b8e944 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:08.632142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14536af7-dc6f-4cf4-a724-41000cca3cab None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:08.641952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 351/710] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:08.643601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dcd51-f77f-4373-8824-166ef174907d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.644589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dcd51-f77f-4373-8824-166ef174907d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.645350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41dcd51-f77f-4373-8824-166ef174907d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.664830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f41dcd51-f77f-4373-8824-166ef174907d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:08.666138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f41dcd51-f77f-4373-8824-166ef174907d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:08.679084 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5f13c-78bb-4475-ad3d-cb9fb191a92e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:08.680175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5f13c-78bb-4475-ad3d-cb9fb191a92e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.680631 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5f13c-78bb-4475-ad3d-cb9fb191a92e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.681064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5f13c-78bb-4475-ad3d-cb9fb191a92e None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.682070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5f13c-78bb-4475-ad3d-cb9fb191a92e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:08.682976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5f13c-78bb-4475-ad3d-cb9fb191a92e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:08.684803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5f13c-78bb-4475-ad3d-cb9fb191a92e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:08.693225 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 360/711] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Sep 6 22:30:08 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:08.694602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5f13c-78bb-4475-ad3d-cb9fb191a92e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:08.762599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 352/712] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:08.763494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dcd51-f77f-4373-8824-166ef174907d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:08.895343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:08.924137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5f13c-78bb-4475-ad3d-cb9fb191a92e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:08.927149 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41dcd51-f77f-4373-8824-166ef174907d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:08.964243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecbf5f-6393-470b-ac05-2f4f37b16151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:08.964452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3e24c-191b-48cd-8e07-1fe6f620b065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:08.965099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3e24c-191b-48cd-8e07-1fe6f620b065 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.965385 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3e24c-191b-48cd-8e07-1fe6f620b065 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.965648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3e24c-191b-48cd-8e07-1fe6f620b065 None admin] PATH_INFO: `/v3/services/network` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.966026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecbf5f-6393-470b-ac05-2f4f37b16151 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:08.966229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3e24c-191b-48cd-8e07-1fe6f620b065 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:08.966731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecbf5f-6393-470b-ac05-2f4f37b16151 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:08.967198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecbf5f-6393-470b-ac05-2f4f37b16151 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:08.967593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3e24c-191b-48cd-8e07-1fe6f620b065 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:08.968569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecbf5f-6393-470b-ac05-2f4f37b16151 None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=a3f142a9e141440cbc854ade4f4135a0) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:08.969106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecbf5f-6393-470b-ac05-2f4f37b16151 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:08.970725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecbf5f-6393-470b-ac05-2f4f37b16151 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:08.977976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 361/713] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:08 2022] GET /identity/v3/role_assignments?user.id=a3f142a9e141440cbc854ade4f4135a0&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 608 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-37a3e24c-191b-48cd-8e07-1fe6f620b065 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:08.979501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:08.980691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:08.982018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 353/714] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:08 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:08.986900 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecbf5f-6393-470b-ac05-2f4f37b16151 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:08.990249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:09.086909 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3e24c-191b-48cd-8e07-1fe6f620b065 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:09.114647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecbf5f-6393-470b-ac05-2f4f37b16151 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:09.155792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc0bc4-ac08-45db-a992-9507a1ea9524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:09.156802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc0bc4-ac08-45db-a992-9507a1ea9524 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:09.157269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc0bc4-ac08-45db-a992-9507a1ea9524 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:09.157723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cc0bc4-ac08-45db-a992-9507a1ea9524 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:09.158553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc0bc4-ac08-45db-a992-9507a1ea9524 None admin] RBAC: Adding query filter params (name=network) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:09.159042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc0bc4-ac08-45db-a992-9507a1ea9524 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:09.161428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cc0bc4-ac08-45db-a992-9507a1ea9524 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:09.166815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 362/715] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:08 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:09.177944 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc0bc4-ac08-45db-a992-9507a1ea9524 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:09.208369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3e24c-191b-48cd-8e07-1fe6f620b065 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:09.258039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e3890-8cb0-4fb4-9db3-b8864051e5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:09.258687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e3890-8cb0-4fb4-9db3-b8864051e5b7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:09.258939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e3890-8cb0-4fb4-9db3-b8864051e5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:09.259212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0e3890-8cb0-4fb4-9db3-b8864051e5b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:09.267646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:09.270008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:09.275068 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e3890-8cb0-4fb4-9db3-b8864051e5b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:09.277628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0e3890-8cb0-4fb4-9db3-b8864051e5b7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-db0e3890-8cb0-4fb4-9db3-b8864051e5b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:09.296570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:09.297679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:09.298564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:09.298564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:09.298564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:09.298564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 354/716] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:09.305018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e3890-8cb0-4fb4-9db3-b8864051e5b7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:09.316568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cc0bc4-ac08-45db-a992-9507a1ea9524 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:09.354869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e01667-2ae0-4d71-8433-3be185ee5113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:09.355765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e01667-2ae0-4d71-8433-3be185ee5113 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:09.356283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e01667-2ae0-4d71-8433-3be185ee5113 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:09.356726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e01667-2ae0-4d71-8433-3be185ee5113 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:09.358179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e01667-2ae0-4d71-8433-3be185ee5113 None admin] RBAC: Adding query filter params (type=network) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:09.358653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e01667-2ae0-4d71-8433-3be185ee5113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:09.360282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e01667-2ae0-4d71-8433-3be185ee5113 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:09.366661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 363/717] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:09 2022] GET /identity/v3/services?type=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:09.429389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0e3890-8cb0-4fb4-9db3-b8864051e5b7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:09.468299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6598dbce-b94a-4386-95d1-949a162d51de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:09.468960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598dbce-b94a-4386-95d1-949a162d51de None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:09.469240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598dbce-b94a-4386-95d1-949a162d51de None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:09.469510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6598dbce-b94a-4386-95d1-949a162d51de None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:09.470185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598dbce-b94a-4386-95d1-949a162d51de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:09.470485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598dbce-b94a-4386-95d1-949a162d51de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:09.471903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6598dbce-b94a-4386-95d1-949a162d51de None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:09.487101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 355/718] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:09.493180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e01667-2ae0-4d71-8433-3be185ee5113 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:09.628550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e01667-2ae0-4d71-8433-3be185ee5113 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:09.659524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:09.667228 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc1e70-3411-49de-a1f3-20b80db59f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:09.667847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc1e70-3411-49de-a1f3-20b80db59f6a None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:09.668123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc1e70-3411-49de-a1f3-20b80db59f6a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:09.668393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc1e70-3411-49de-a1f3-20b80db59f6a None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:09.669099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc1e70-3411-49de-a1f3-20b80db59f6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:09.670520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc1e70-3411-49de-a1f3-20b80db59f6a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:09.696722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 364/719] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:30:09 2022] POST /identity/v3/users => generated 254 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:10.531424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250ccf-22ae-43c7-b6b2-e006b54adfcf None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:10.531944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250ccf-22ae-43c7-b6b2-e006b54adfcf None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:10.531944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da250ccf-22ae-43c7-b6b2-e006b54adfcf None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:10.533578 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 356/720] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:10.539470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97b29a-aef5-4ceb-84a3-e5b00670edb3 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:10.539894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97b29a-aef5-4ceb-84a3-e5b00670edb3 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:10.540063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97b29a-aef5-4ceb-84a3-e5b00670edb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:10.566324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-0f97b29a-aef5-4ceb-84a3-e5b00670edb3 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:10.567536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-0f97b29a-aef5-4ceb-84a3-e5b00670edb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:10.654049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 365/721] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:11.037579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da250ccf-22ae-43c7-b6b2-e006b54adfcf None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:11.102894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:11.174386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da250ccf-22ae-43c7-b6b2-e006b54adfcf None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:11.218852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390ba22e-4740-4322-b864-571e2aaa1415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:11.219480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ba22e-4740-4322-b864-571e2aaa1415 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:11.219754 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ba22e-4740-4322-b864-571e2aaa1415 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:11.220019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390ba22e-4740-4322-b864-571e2aaa1415 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:11.235128 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390ba22e-4740-4322-b864-571e2aaa1415 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-390ba22e-4740-4322-b864-571e2aaa1415 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:11.241330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:11.242281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:11.243124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 357/722] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:30:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:11.249864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97b29a-aef5-4ceb-84a3-e5b00670edb3 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:11.402285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6f0ce-5ca8-4442-9df5-365625781d63 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:11.402560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6f0ce-5ca8-4442-9df5-365625781d63 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:11.402833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6f0ce-5ca8-4442-9df5-365625781d63 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:11.404419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 358/723] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:11.405804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97b29a-aef5-4ceb-84a3-e5b00670edb3 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:11.414918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d82aa5-bf64-4e02-b541-5f2533c1b66f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:11.415445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d82aa5-bf64-4e02-b541-5f2533c1b66f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:11.415907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d82aa5-bf64-4e02-b541-5f2533c1b66f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:11.442672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:11.446445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c1d82aa5-bf64-4e02-b541-5f2533c1b66f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:11.447533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c1d82aa5-bf64-4e02-b541-5f2533c1b66f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:11.470288 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfb2f9-54c2-4de1-b67f-314b8a4cb9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:11.471282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfb2f9-54c2-4de1-b67f-314b8a4cb9d3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:11.471563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfb2f9-54c2-4de1-b67f-314b8a4cb9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:11.471834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cfb2f9-54c2-4de1-b67f-314b8a4cb9d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:11.473372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfb2f9-54c2-4de1-b67f-314b8a4cb9d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:11.473830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfb2f9-54c2-4de1-b67f-314b8a4cb9d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:11.475195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cfb2f9-54c2-4de1-b67f-314b8a4cb9d3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:11.493906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 366/724] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:30:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:11.497936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfb2f9-54c2-4de1-b67f-314b8a4cb9d3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:11.594720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 359/725] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 06 22:30:11.752838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cfb2f9-54c2-4de1-b67f-314b8a4cb9d3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:11.812960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaaf5a-5d38-4fea-bde5-a3aba921acb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:11.813596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaaf5a-5d38-4fea-bde5-a3aba921acb9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:11.813875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaaf5a-5d38-4fea-bde5-a3aba921acb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:11.814143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaaf5a-5d38-4fea-bde5-a3aba921acb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:11.833693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaaf5a-5d38-4fea-bde5-a3aba921acb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:11.841771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaaf5a-5d38-4fea-bde5-a3aba921acb9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-cacaaf5a-5d38-4fea-bde5-a3aba921acb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:11.859598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:11.860605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:11.861548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:11.861548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:11.861548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:11.864264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 367/726] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 367 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:11.873056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d82aa5-bf64-4e02-b541-5f2533c1b66f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:11.960346 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaaf5a-5d38-4fea-bde5-a3aba921acb9 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:11.985841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:12.055051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d82aa5-bf64-4e02-b541-5f2533c1b66f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:12.073740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:12.096925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b776d97c-eea1-4dc4-9ca0-5b5dba57fd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:12.098723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d97c-eea1-4dc4-9ca0-5b5dba57fd24 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:12.099204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d97c-eea1-4dc4-9ca0-5b5dba57fd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:12.102507 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b776d97c-eea1-4dc4-9ca0-5b5dba57fd24 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:12.105030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776d97c-eea1-4dc4-9ca0-5b5dba57fd24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:12.105832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776d97c-eea1-4dc4-9ca0-5b5dba57fd24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:12.106761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b776d97c-eea1-4dc4-9ca0-5b5dba57fd24 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:12.140105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 360/727] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:12.141762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1235599-bb44-42e5-beb4-7b7598f90af5 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:12.142651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1235599-bb44-42e5-beb4-7b7598f90af5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:12.142924 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1235599-bb44-42e5-beb4-7b7598f90af5 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:12.147477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 361/728] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:12.148266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1235599-bb44-42e5-beb4-7b7598f90af5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:12.212918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaaf5a-5d38-4fea-bde5-a3aba921acb9 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:12.263155 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0bb1da-a99f-476d-9869-98188e1ec257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:12.263853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bb1da-a99f-476d-9869-98188e1ec257 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:12.264132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bb1da-a99f-476d-9869-98188e1ec257 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:12.264401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0bb1da-a99f-476d-9869-98188e1ec257 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:12.265087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bb1da-a99f-476d-9869-98188e1ec257 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:12.266443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0bb1da-a99f-476d-9869-98188e1ec257 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:12.289330 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 368/729] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:30:11 2022] POST /identity/v3/services => generated 251 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:30:12.290614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1dd5-2cdc-4fa6-a46f-6188eae02fac None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:12.290900 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1dd5-2cdc-4fa6-a46f-6188eae02fac None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:12.291176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd1dd5-2cdc-4fa6-a46f-6188eae02fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:12.303771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:12.317864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1235599-bb44-42e5-beb4-7b7598f90af5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:12.335366 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e3fd1dd5-2cdc-4fa6-a46f-6188eae02fac None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:12.337286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e3fd1dd5-2cdc-4fa6-a46f-6188eae02fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:12.379112 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c24b5-da19-459f-8a0a-16ce0f9e5c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:12.381525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c24b5-da19-459f-8a0a-16ce0f9e5c7d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:12.381821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c24b5-da19-459f-8a0a-16ce0f9e5c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:12.383466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c24b5-da19-459f-8a0a-16ce0f9e5c7d None admin] PATH_INFO: `/v3/users/nova` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:12.399507 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c24b5-da19-459f-8a0a-16ce0f9e5c7d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:12.401902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c24b5-da19-459f-8a0a-16ce0f9e5c7d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a53c24b5-da19-459f-8a0a-16ce0f9e5c7d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:12.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:12.431330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 362/730] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Sep 6 22:30:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5996b1-69ef-4b5d-ba62-1fb092251873 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:12.437458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5996b1-69ef-4b5d-ba62-1fb092251873 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:12.438163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5996b1-69ef-4b5d-ba62-1fb092251873 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:12.438163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 363/731] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:12.438163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5996b1-69ef-4b5d-ba62-1fb092251873 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:12.483600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 369/732] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:12.488962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ab037-64d8-4b4a-bd96-38f7f5386c88 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:12.488962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ab037-64d8-4b4a-bd96-38f7f5386c88 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:12.488962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196ab037-64d8-4b4a-bd96-38f7f5386c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:12.514743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-196ab037-64d8-4b4a-bd96-38f7f5386c88 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:12.516468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-196ab037-64d8-4b4a-bd96-38f7f5386c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:12.538807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:12.596576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5996b1-69ef-4b5d-ba62-1fb092251873 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:12.611997 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 370/733] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:12.639875 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee277a-5025-4b6d-b0fd-e2b59ed6257a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:12.640482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee277a-5025-4b6d-b0fd-e2b59ed6257a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:12.640777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee277a-5025-4b6d-b0fd-e2b59ed6257a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:12.641081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee277a-5025-4b6d-b0fd-e2b59ed6257a None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:12.641810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee277a-5025-4b6d-b0fd-e2b59ed6257a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:12.642112 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee277a-5025-4b6d-b0fd-e2b59ed6257a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:12.643639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee277a-5025-4b6d-b0fd-e2b59ed6257a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:12.662331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 364/734] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Sep 6 22:30:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:12.667921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ab037-64d8-4b4a-bd96-38f7f5386c88 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:12.807296 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196ab037-64d8-4b4a-bd96-38f7f5386c88 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:12.813425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b0f5c-f1a4-4a77-b28c-e04396e11e43 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:12.813775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b0f5c-f1a4-4a77-b28c-e04396e11e43 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:12.814038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247b0f5c-f1a4-4a77-b28c-e04396e11e43 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:12.815659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 365/735] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:12.819548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b0f5c-f1a4-4a77-b28c-e04396e11e43 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:12.837213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:12.865023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce69dd1-5520-49b9-9b68-5d730d17612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:12.865657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69dd1-5520-49b9-9b68-5d730d17612e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:12.865935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69dd1-5520-49b9-9b68-5d730d17612e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:12.866206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce69dd1-5520-49b9-9b68-5d730d17612e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:12.876964 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce69dd1-5520-49b9-9b68-5d730d17612e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:12.878440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce69dd1-5520-49b9-9b68-5d730d17612e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-dce69dd1-5520-49b9-9b68-5d730d17612e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:12.898139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:12.899121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:12.900036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:12.900036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:12.900036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:12.900036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 371/736] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:12.905003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627bac3-f480-4ac7-8061-9a3812f498f1 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:12.906763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627bac3-f480-4ac7-8061-9a3812f498f1 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:12.906763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d627bac3-f480-4ac7-8061-9a3812f498f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:12.913257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:12.929495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d627bac3-f480-4ac7-8061-9a3812f498f1 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:12.931112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d627bac3-f480-4ac7-8061-9a3812f498f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:12.937241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247b0f5c-f1a4-4a77-b28c-e04396e11e43 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:13.006405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 372/737] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:12 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 06 22:30:13.007843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627bac3-f480-4ac7-8061-9a3812f498f1 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:13.089642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4eada-0611-42ed-b19c-10c48e6cb2b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:13.090182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4eada-0611-42ed-b19c-10c48e6cb2b6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:13.090457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4eada-0611-42ed-b19c-10c48e6cb2b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:13.090726 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e4eada-0611-42ed-b19c-10c48e6cb2b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:13.101981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e4eada-0611-42ed-b19c-10c48e6cb2b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f2e4eada-0611-42ed-b19c-10c48e6cb2b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:13.107049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:13.108074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:13.108913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 366/738] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:30:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:13.109616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4eada-0611-42ed-b19c-10c48e6cb2b6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:13.141913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d627bac3-f480-4ac7-8061-9a3812f498f1 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:13.177974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:13.185215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532a7a9-c9b1-42ab-9f96-181df2278143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:13.185894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:13.187259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532a7a9-c9b1-42ab-9f96-181df2278143 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:13.187735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532a7a9-c9b1-42ab-9f96-181df2278143 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:13.188307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b532a7a9-c9b1-42ab-9f96-181df2278143 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:13.189345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532a7a9-c9b1-42ab-9f96-181df2278143 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:13.189876 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532a7a9-c9b1-42ab-9f96-181df2278143 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:13.191579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b532a7a9-c9b1-42ab-9f96-181df2278143 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:13.209267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 373/739] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:13.210267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532a7a9-c9b1-42ab-9f96-181df2278143 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:13.211947 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e4eada-0611-42ed-b19c-10c48e6cb2b6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:13.248788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468e4e7-eb36-415c-bb5f-b4c24103f363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:13.252792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468e4e7-eb36-415c-bb5f-b4c24103f363 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:13.252792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468e4e7-eb36-415c-bb5f-b4c24103f363 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:13.252792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468e4e7-eb36-415c-bb5f-b4c24103f363 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:13.252792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468e4e7-eb36-415c-bb5f-b4c24103f363 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:13.252792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468e4e7-eb36-415c-bb5f-b4c24103f363 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:13.271678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 367/740] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:30:13 2022] POST /identity/v3/services => generated 240 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:13.272566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468e4e7-eb36-415c-bb5f-b4c24103f363 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:13.329640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b532a7a9-c9b1-42ab-9f96-181df2278143 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:13.390172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5766b18-b791-4bf3-8739-85019360adf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:13.390817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5766b18-b791-4bf3-8739-85019360adf2 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:13.392302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5766b18-b791-4bf3-8739-85019360adf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:13.392594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5766b18-b791-4bf3-8739-85019360adf2 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:13.393305 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5766b18-b791-4bf3-8739-85019360adf2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:13.393605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5766b18-b791-4bf3-8739-85019360adf2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:13.395751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5766b18-b791-4bf3-8739-85019360adf2 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:13.402751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468e4e7-eb36-415c-bb5f-b4c24103f363 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:13.406755 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 374/741] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:30:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:13.406926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5766b18-b791-4bf3-8739-85019360adf2 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:13.458275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d0a7c-799f-4ae9-99f3-7cadf84649ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:13.459766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d0a7c-799f-4ae9-99f3-7cadf84649ba None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:13.460042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d0a7c-799f-4ae9-99f3-7cadf84649ba None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:13.460308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5d0a7c-799f-4ae9-99f3-7cadf84649ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:13.465714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:13.469113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:13.474038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d0a7c-799f-4ae9-99f3-7cadf84649ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:13.475727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5d0a7c-799f-4ae9-99f3-7cadf84649ba None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-4f5d0a7c-799f-4ae9-99f3-7cadf84649ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:13.493423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:13.494384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:13.495280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:13.495280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:13.495280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:13.495280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:13.495280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:13.495280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:13.495280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:13.495280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:13.495280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 368/742] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:13.495768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d0a7c-799f-4ae9-99f3-7cadf84649ba None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:13.546969 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5766b18-b791-4bf3-8739-85019360adf2 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:13.598534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d618011-a2dd-417d-a5be-bdffe69fe685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:13.599163 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d618011-a2dd-417d-a5be-bdffe69fe685 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:13.599436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d618011-a2dd-417d-a5be-bdffe69fe685 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:13.599704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d618011-a2dd-417d-a5be-bdffe69fe685 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:13.610507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d618011-a2dd-417d-a5be-bdffe69fe685 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-7d618011-a2dd-417d-a5be-bdffe69fe685 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:13.615768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:13.616733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:13.617606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:13.620900 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 375/743] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:30:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:13.622315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d618011-a2dd-417d-a5be-bdffe69fe685 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:13.633982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5d0a7c-799f-4ae9-99f3-7cadf84649ba None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:13.674166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ccb12-b582-4fdf-88e2-1033d8c4fbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:13.675149 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ccb12-b582-4fdf-88e2-1033d8c4fbe2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:13.675683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ccb12-b582-4fdf-88e2-1033d8c4fbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:13.676167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ccb12-b582-4fdf-88e2-1033d8c4fbe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:13.688553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ccb12-b582-4fdf-88e2-1033d8c4fbe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:13.690322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ccb12-b582-4fdf-88e2-1033d8c4fbe2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-874ccb12-b582-4fdf-88e2-1033d8c4fbe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:13.706197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:13.707235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:13.708129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:13.708129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:13.708129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:13.710190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 369/744] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:13.711205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ccb12-b582-4fdf-88e2-1033d8c4fbe2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:13.736487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d618011-a2dd-417d-a5be-bdffe69fe685 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:13.747861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:13.762556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:13.774979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798749f-4825-4931-bdfd-fe901042cad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:13.775653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798749f-4825-4931-bdfd-fe901042cad8 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:13.775918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798749f-4825-4931-bdfd-fe901042cad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:13.776184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0798749f-4825-4931-bdfd-fe901042cad8 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:13.777003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798749f-4825-4931-bdfd-fe901042cad8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:13.777434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798749f-4825-4931-bdfd-fe901042cad8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:13.778874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0798749f-4825-4931-bdfd-fe901042cad8 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:13.801026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 376/745] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:13.802214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798749f-4825-4931-bdfd-fe901042cad8 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:13.827766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ccb12-b582-4fdf-88e2-1033d8c4fbe2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:13.879121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0f3fe-94ab-41dc-b4e3-976d934ec75e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:13.880570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0f3fe-94ab-41dc-b4e3-976d934ec75e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:13.881178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0f3fe-94ab-41dc-b4e3-976d934ec75e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:13.881670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a0f3fe-94ab-41dc-b4e3-976d934ec75e None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:13.882808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0f3fe-94ab-41dc-b4e3-976d934ec75e None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:13.883353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0f3fe-94ab-41dc-b4e3-976d934ec75e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:13.884967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a0f3fe-94ab-41dc-b4e3-976d934ec75e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:13.898250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 370/746] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:30:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:13.899915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0f3fe-94ab-41dc-b4e3-976d934ec75e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:13.920537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0798749f-4825-4931-bdfd-fe901042cad8 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:13.958761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7efdc2-6ae0-4ab4-b44d-da54526748ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:13.959283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7efdc2-6ae0-4ab4-b44d-da54526748ea None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:13.959604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7efdc2-6ae0-4ab4-b44d-da54526748ea None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:13.959927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7efdc2-6ae0-4ab4-b44d-da54526748ea None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:13.960600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7efdc2-6ae0-4ab4-b44d-da54526748ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:13.960947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7efdc2-6ae0-4ab4-b44d-da54526748ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:13.962417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7efdc2-6ae0-4ab4-b44d-da54526748ea None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:13.976707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 377/747] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:13.977928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7efdc2-6ae0-4ab4-b44d-da54526748ea None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:14.011439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:14.023337 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a0f3fe-94ab-41dc-b4e3-976d934ec75e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:14.033335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:14.061436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c948de4-9913-4228-ab6e-fac867003d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:14.062478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c948de4-9913-4228-ab6e-fac867003d51 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:14.063000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c948de4-9913-4228-ab6e-fac867003d51 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:14.063477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c948de4-9913-4228-ab6e-fac867003d51 None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/b16947313dad4d59a97cc8b016ee54c1/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:14.065800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c948de4-9913-4228-ab6e-fac867003d51 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=b16947313dad4d59a97cc8b016ee54c1, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:14.067610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c948de4-9913-4228-ab6e-fac867003d51 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:14.083044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 371/748] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:30:13 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/b16947313dad4d59a97cc8b016ee54c1/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:30:14.084284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c948de4-9913-4228-ab6e-fac867003d51 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:14.107395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7efdc2-6ae0-4ab4-b44d-da54526748ea None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:14.192590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162492e5-45a0-4cf7-8915-903cac950fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:14.193262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162492e5-45a0-4cf7-8915-903cac950fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:14.193535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162492e5-45a0-4cf7-8915-903cac950fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:14.193867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162492e5-45a0-4cf7-8915-903cac950fb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:14.208322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162492e5-45a0-4cf7-8915-903cac950fb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:14.210227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162492e5-45a0-4cf7-8915-903cac950fb0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:14.224484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c948de4-9913-4228-ab6e-fac867003d51 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-162492e5-45a0-4cf7-8915-903cac950fb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:14.233558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:14.234620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:14.235859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:14.235859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:14.235859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:14.235859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 378/749] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:14.246500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162492e5-45a0-4cf7-8915-903cac950fb0 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:14.327934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322d4e3-672a-419b-883f-860fc679864f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:14.328745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322d4e3-672a-419b-883f-860fc679864f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:14.329106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322d4e3-672a-419b-883f-860fc679864f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:14.329664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2322d4e3-672a-419b-883f-860fc679864f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:14.339562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322d4e3-672a-419b-883f-860fc679864f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:14.341286 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2322d4e3-672a-419b-883f-860fc679864f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2322d4e3-672a-419b-883f-860fc679864f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:14.356739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:14.357747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:14.358681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:14.358681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:14.358681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:14.358681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:14.358681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:14.358681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:14.358681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:14.358681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:14.358681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:30:14.358681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:14.359633 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 372/750] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:30:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:14.368724 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322d4e3-672a-419b-883f-860fc679864f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:14.408719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:14.408958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:14.433458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162492e5-45a0-4cf7-8915-903cac950fb0 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:14.478504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68a73b-f808-42d9-8f84-d036ca9a1479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:14.479529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68a73b-f808-42d9-8f84-d036ca9a1479 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:14.479984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68a73b-f808-42d9-8f84-d036ca9a1479 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:14.480449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b68a73b-f808-42d9-8f84-d036ca9a1479 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:14.481329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68a73b-f808-42d9-8f84-d036ca9a1479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:14.481813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68a73b-f808-42d9-8f84-d036ca9a1479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:14.483349 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b68a73b-f808-42d9-8f84-d036ca9a1479 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:14.496746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 379/751] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:14.505621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68a73b-f808-42d9-8f84-d036ca9a1479 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:14.514045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2322d4e3-672a-419b-883f-860fc679864f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:14.586139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba8edb-fe5b-4c5e-9046-c670e5ade81a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:14.586759 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba8edb-fe5b-4c5e-9046-c670e5ade81a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:14.587032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba8edb-fe5b-4c5e-9046-c670e5ade81a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:14.587301 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ba8edb-fe5b-4c5e-9046-c670e5ade81a None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:14.588023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba8edb-fe5b-4c5e-9046-c670e5ade81a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:14.588321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba8edb-fe5b-4c5e-9046-c670e5ade81a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:14.601956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ba8edb-fe5b-4c5e-9046-c670e5ade81a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:14.633801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 373/752] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Sep 6 22:30:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:14.639363 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba8edb-fe5b-4c5e-9046-c670e5ade81a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:14.656777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b68a73b-f808-42d9-8f84-d036ca9a1479 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:14.708785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1bcad-3f0e-4a15-8e2e-98325294dbe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:14.708785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1bcad-3f0e-4a15-8e2e-98325294dbe6 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:14.708785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1bcad-3f0e-4a15-8e2e-98325294dbe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:14.708785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba1bcad-3f0e-4a15-8e2e-98325294dbe6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:14.724934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1bcad-3f0e-4a15-8e2e-98325294dbe6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:14.726406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba1bcad-3f0e-4a15-8e2e-98325294dbe6 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-7ba1bcad-3f0e-4a15-8e2e-98325294dbe6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:14.756679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:14.759105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:14.761325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:14.761325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:14.761325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:14.761325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:14.761325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:14.761325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:14.761325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:14.761325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:14.761325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:14.761325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:14.768775 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 380/753] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Sep 6 22:30:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:14.774660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1bcad-3f0e-4a15-8e2e-98325294dbe6 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:14.778737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:14.798115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:14.850072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ba8edb-fe5b-4c5e-9046-c670e5ade81a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:14.897756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba1bcad-3f0e-4a15-8e2e-98325294dbe6 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:14.929820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d052bb-c462-451a-b4d1-5b3c1d08cba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:14.930799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d052bb-c462-451a-b4d1-5b3c1d08cba2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:14.931272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d052bb-c462-451a-b4d1-5b3c1d08cba2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:14.931994 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d052bb-c462-451a-b4d1-5b3c1d08cba2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:14.942922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c5b238-63b3-4f70-a3dc-919ea912171c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:14.943547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5b238-63b3-4f70-a3dc-919ea912171c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:14.943798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5b238-63b3-4f70-a3dc-919ea912171c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:14.944077 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c5b238-63b3-4f70-a3dc-919ea912171c None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:14.944852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5b238-63b3-4f70-a3dc-919ea912171c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:14.945151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5b238-63b3-4f70-a3dc-919ea912171c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:14.946858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c5b238-63b3-4f70-a3dc-919ea912171c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:14.968384 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 381/754] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Sep 6 22:30:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:14.969734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c96bca-1eaf-4255-9c83-b88c2752b7b8 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:14.970079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c96bca-1eaf-4255-9c83-b88c2752b7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:14.970355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c96bca-1eaf-4255-9c83-b88c2752b7b8 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:14.976851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 382/755] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:14.976851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c96bca-1eaf-4255-9c83-b88c2752b7b8 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:15.012015 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d052bb-c462-451a-b4d1-5b3c1d08cba2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:15.013772 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d052bb-c462-451a-b4d1-5b3c1d08cba2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:15.048407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:15.099439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c96bca-1eaf-4255-9c83-b88c2752b7b8 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:15.103026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:15.119235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 374/756] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:30:15.120786 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6a929-a8fe-4fb6-8909-618ec645f8d7 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:15.121249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6a929-a8fe-4fb6-8909-618ec645f8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:15.121823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c6a929-a8fe-4fb6-8909-618ec645f8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:15.140531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-07c6a929-a8fe-4fb6-8909-618ec645f8d7 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:15.140896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d5ea6-0460-4cbd-8556-a15030a85df2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:15.141514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d5ea6-0460-4cbd-8556-a15030a85df2 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:15.141794 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d5ea6-0460-4cbd-8556-a15030a85df2 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:15.142170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60d5ea6-0460-4cbd-8556-a15030a85df2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:15.142926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-07c6a929-a8fe-4fb6-8909-618ec645f8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:15.228810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d5ea6-0460-4cbd-8556-a15030a85df2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:15.230638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60d5ea6-0460-4cbd-8556-a15030a85df2 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:15.240790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 375/757] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:15.240790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6a929-a8fe-4fb6-8909-618ec645f8d7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:15.352697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:15.438520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c6a929-a8fe-4fb6-8909-618ec645f8d7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:15.439975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 383/758] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:14 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:30:15.440890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d5ea6-0460-4cbd-8556-a15030a85df2 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:15.470066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:15.511820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75eb7e-3da7-44e2-9ce9-be3104d64364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:15.514718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75eb7e-3da7-44e2-9ce9-be3104d64364 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:15.514995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75eb7e-3da7-44e2-9ce9-be3104d64364 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:15.515272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d75eb7e-3da7-44e2-9ce9-be3104d64364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:15.544505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75eb7e-3da7-44e2-9ce9-be3104d64364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:15.546252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d75eb7e-3da7-44e2-9ce9-be3104d64364 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2d75eb7e-3da7-44e2-9ce9-be3104d64364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:15.565843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:15.567250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:15.568123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:15.568123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:15.568123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:15.568123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 376/759] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:15.570642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75eb7e-3da7-44e2-9ce9-be3104d64364 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:15.594933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60d5ea6-0460-4cbd-8556-a15030a85df2 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:15.638656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2323b-9905-4a3d-a755-28ae89552232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:15.639210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2323b-9905-4a3d-a755-28ae89552232 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:15.639489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2323b-9905-4a3d-a755-28ae89552232 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:15.639759 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae2323b-9905-4a3d-a755-28ae89552232 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:15.650525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2323b-9905-4a3d-a755-28ae89552232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:15.651948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae2323b-9905-4a3d-a755-28ae89552232 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-7ae2323b-9905-4a3d-a755-28ae89552232 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:15.666877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:15.667906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:15.669541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:15.669541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:15.669541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:15.669541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 384/760] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:15.669541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2323b-9905-4a3d-a755-28ae89552232 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:15.687795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:15.695565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d75eb7e-3da7-44e2-9ce9-be3104d64364 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:15.734912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f39fa-f757-4df7-b38e-c38a5457f1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:15.735549 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f39fa-f757-4df7-b38e-c38a5457f1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:15.735824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f39fa-f757-4df7-b38e-c38a5457f1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:15.736093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f39fa-f757-4df7-b38e-c38a5457f1c2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:15.736676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f39fa-f757-4df7-b38e-c38a5457f1c2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:15.738793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f39fa-f757-4df7-b38e-c38a5457f1c2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-da8f39fa-f757-4df7-b38e-c38a5457f1c2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:15.748489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:15.749586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:15.750431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 377/761] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Sep 6 22:30:15 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:15.751233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b1dab-a5f3-4bca-b64a-f3bddcbb2873 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:15.751233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b1dab-a5f3-4bca-b64a-f3bddcbb2873 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:15.751354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b1dab-a5f3-4bca-b64a-f3bddcbb2873 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:15.757009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 378/762] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:15.757009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b1dab-a5f3-4bca-b64a-f3bddcbb2873 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:15.790591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:15.791816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae2323b-9905-4a3d-a755-28ae89552232 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:15.829481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374debc-50af-4dbb-97e9-4d3839905db5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:15.830194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374debc-50af-4dbb-97e9-4d3839905db5 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:15.830468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374debc-50af-4dbb-97e9-4d3839905db5 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:15.830846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a374debc-50af-4dbb-97e9-4d3839905db5 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:15.831512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374debc-50af-4dbb-97e9-4d3839905db5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:15.832326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374debc-50af-4dbb-97e9-4d3839905db5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:15.833727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a374debc-50af-4dbb-97e9-4d3839905db5 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:15.848828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 385/763] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:15.849979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374debc-50af-4dbb-97e9-4d3839905db5 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:15.866487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b1dab-a5f3-4bca-b64a-f3bddcbb2873 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:15.902838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82ad55f-4de0-4442-8e16-738c8e6ffcfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:15.903394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad55f-4de0-4442-8e16-738c8e6ffcfd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:15.903853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad55f-4de0-4442-8e16-738c8e6ffcfd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:15.903980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82ad55f-4de0-4442-8e16-738c8e6ffcfd None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:15.904549 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ad55f-4de0-4442-8e16-738c8e6ffcfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:15.904875 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ad55f-4de0-4442-8e16-738c8e6ffcfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:15.906283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82ad55f-4de0-4442-8e16-738c8e6ffcfd None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:15.922444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 379/764] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:15.922444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00dd62-579a-4337-9863-a0d751fa853c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:15.922444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00dd62-579a-4337-9863-a0d751fa853c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:15.922444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00dd62-579a-4337-9863-a0d751fa853c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:15.930546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:15.943023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-da00dd62-579a-4337-9863-a0d751fa853c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:15.944443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-da00dd62-579a-4337-9863-a0d751fa853c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:15.965704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a374debc-50af-4dbb-97e9-4d3839905db5 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:16.001227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c158a11-45ae-4350-b1d4-7d82f2a7b5ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:16.001888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158a11-45ae-4350-b1d4-7d82f2a7b5ea None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.002104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158a11-45ae-4350-b1d4-7d82f2a7b5ea None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.002374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c158a11-45ae-4350-b1d4-7d82f2a7b5ea None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.003002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c158a11-45ae-4350-b1d4-7d82f2a7b5ea None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:16.003297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c158a11-45ae-4350-b1d4-7d82f2a7b5ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:16.005238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c158a11-45ae-4350-b1d4-7d82f2a7b5ea None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:16.010193 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 386/765] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Sep 6 22:30:15 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:16.015482 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c158a11-45ae-4350-b1d4-7d82f2a7b5ea None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:16.024779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 380/766] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:15 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:16.024779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00dd62-579a-4337-9863-a0d751fa853c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:16.058534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:16.134675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c158a11-45ae-4350-b1d4-7d82f2a7b5ea None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:16.140922 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00dd62-579a-4337-9863-a0d751fa853c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:16.172990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc907a0-63f0-495b-9bc0-b2cac25197fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:16.173640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc907a0-63f0-495b-9bc0-b2cac25197fd None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.173986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc907a0-63f0-495b-9bc0-b2cac25197fd None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.174331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc907a0-63f0-495b-9bc0-b2cac25197fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.177434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:16.182770 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc907a0-63f0-495b-9bc0-b2cac25197fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:16.184586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc907a0-63f0-495b-9bc0-b2cac25197fd None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:16.187259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fc593-b0fb-4fcc-bc2f-79a90fda042d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:16.188163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fc593-b0fb-4fcc-bc2f-79a90fda042d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.188587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fc593-b0fb-4fcc-bc2f-79a90fda042d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.189166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fc593-b0fb-4fcc-bc2f-79a90fda042d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.196331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5fc593-b0fb-4fcc-bc2f-79a90fda042d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-6bc907a0-63f0-495b-9bc0-b2cac25197fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:16.197831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:16.199044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:16.200106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:16.200106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:16.200106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:16.200106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:16.200106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:16.200106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:16.200106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:16.200106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:16.200106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 387/767] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:16.200106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc907a0-63f0-495b-9bc0-b2cac25197fd None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:16.201028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5fc593-b0fb-4fcc-bc2f-79a90fda042d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-3f5fc593-b0fb-4fcc-bc2f-79a90fda042d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:16.214250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:16.215367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:16.216487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:16.216487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:16.216487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:16.216487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:16.216487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:16.216487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:16.216487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:16.216487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:16.218648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 381/768] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:16.219609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fc593-b0fb-4fcc-bc2f-79a90fda042d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:16.314303 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:16.319499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc907a0-63f0-495b-9bc0-b2cac25197fd None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:16.335788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fc593-b0fb-4fcc-bc2f-79a90fda042d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:16.356467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a15339-b7f9-47b5-8ce8-13720daf6662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:16.357194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a15339-b7f9-47b5-8ce8-13720daf6662 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.357533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a15339-b7f9-47b5-8ce8-13720daf6662 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.357878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a15339-b7f9-47b5-8ce8-13720daf6662 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.358575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a15339-b7f9-47b5-8ce8-13720daf6662 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:16.358882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a15339-b7f9-47b5-8ce8-13720daf6662 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:16.360227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a15339-b7f9-47b5-8ce8-13720daf6662 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:16.370200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 388/769] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Sep 6 22:30:16 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:16.370200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a15339-b7f9-47b5-8ce8-13720daf6662 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:16.378145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f047ef-d130-409e-9ae3-a6060940eb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:16.378880 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f047ef-d130-409e-9ae3-a6060940eb48 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.379227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f047ef-d130-409e-9ae3-a6060940eb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.379928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f047ef-d130-409e-9ae3-a6060940eb48 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.380628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f047ef-d130-409e-9ae3-a6060940eb48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:16.380963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f047ef-d130-409e-9ae3-a6060940eb48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:16.382433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f047ef-d130-409e-9ae3-a6060940eb48 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:16.396570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 382/770] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:16.401958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f047ef-d130-409e-9ae3-a6060940eb48 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:16.435550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:16.484970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a15339-b7f9-47b5-8ce8-13720daf6662 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:16.509768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f047ef-d130-409e-9ae3-a6060940eb48 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:16.523931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fb3c6-da10-434e-ba8c-31768952da58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:16.524543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fb3c6-da10-434e-ba8c-31768952da58 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.524853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fb3c6-da10-434e-ba8c-31768952da58 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.525125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0fb3c6-da10-434e-ba8c-31768952da58 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.525806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fb3c6-da10-434e-ba8c-31768952da58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:16.526111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fb3c6-da10-434e-ba8c-31768952da58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:16.527580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0fb3c6-da10-434e-ba8c-31768952da58 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:16.544456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 389/771] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:16.544456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fb3c6-da10-434e-ba8c-31768952da58 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:16.554691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6056b5b-9967-4b12-aa93-04f97945b1ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:16.555690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056b5b-9967-4b12-aa93-04f97945b1ad None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.556020 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056b5b-9967-4b12-aa93-04f97945b1ad None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.556294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6056b5b-9967-4b12-aa93-04f97945b1ad None admin] PATH_INFO: `/v3/services/network` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.557272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6056b5b-9967-4b12-aa93-04f97945b1ad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:16.558941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6056b5b-9967-4b12-aa93-04f97945b1ad None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e6056b5b-9967-4b12-aa93-04f97945b1ad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:16.569588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:16.570648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:16.571491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 383/772] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:16 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:16.572271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6056b5b-9967-4b12-aa93-04f97945b1ad None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:16.585776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:16.655228 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0fb3c6-da10-434e-ba8c-31768952da58 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:16.674326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:16.681423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6056b5b-9967-4b12-aa93-04f97945b1ad None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:16.690682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b25177-7d3b-4d19-acdc-5a2e555b57c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:16.691821 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b25177-7d3b-4d19-acdc-5a2e555b57c1 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.692402 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b25177-7d3b-4d19-acdc-5a2e555b57c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.692677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b25177-7d3b-4d19-acdc-5a2e555b57c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.693396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b25177-7d3b-4d19-acdc-5a2e555b57c1 None admin] RBAC: Adding query filter params (interface=public, service_id=f7759d05d3754dff951c57125ece88c0, region_id=RegionOne) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:16.693699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b25177-7d3b-4d19-acdc-5a2e555b57c1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:16.695061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b25177-7d3b-4d19-acdc-5a2e555b57c1 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:16.702418 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 390/773] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Sep 6 22:30:16 2022] GET /identity/v3/endpoints?service_id=f7759d05d3754dff951c57125ece88c0&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:16.703199 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b25177-7d3b-4d19-acdc-5a2e555b57c1 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:16.722841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4ca33-443b-45ae-bcda-a124ea867dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:16.724069 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ca33-443b-45ae-bcda-a124ea867dcb None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.724415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ca33-443b-45ae-bcda-a124ea867dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.724786 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e4ca33-443b-45ae-bcda-a124ea867dcb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.725583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4ca33-443b-45ae-bcda-a124ea867dcb None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=a3f142a9e141440cbc854ade4f4135a0) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:16.725955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4ca33-443b-45ae-bcda-a124ea867dcb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:16.727624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e4ca33-443b-45ae-bcda-a124ea867dcb None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:16.733712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 384/774] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:16 2022] GET /identity/v3/role_assignments?user.id=a3f142a9e141440cbc854ade4f4135a0&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:16.734574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4ca33-443b-45ae-bcda-a124ea867dcb None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:16.827188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:16.846860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b25177-7d3b-4d19-acdc-5a2e555b57c1 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:16.878317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e4ca33-443b-45ae-bcda-a124ea867dcb None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:16.901504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72678d26-764d-4303-b408-e036ed47c125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:16.902040 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678d26-764d-4303-b408-e036ed47c125 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.902443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678d26-764d-4303-b408-e036ed47c125 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.902771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678d26-764d-4303-b408-e036ed47c125 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.903621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72678d26-764d-4303-b408-e036ed47c125 None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=7da74f16cb9b4b15a902947103a25cf6) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:16.904035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72678d26-764d-4303-b408-e036ed47c125 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:16.905614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72678d26-764d-4303-b408-e036ed47c125 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:16.916678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 391/775] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:16 2022] GET /identity/v3/role_assignments?user.id=7da74f16cb9b4b15a902947103a25cf6&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:16.917891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bc849-ea51-4fc6-bda2-60f0a112b9cc None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.921081 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bc849-ea51-4fc6-bda2-60f0a112b9cc None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.921396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184bc849-ea51-4fc6-bda2-60f0a112b9cc None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.922949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 392/776] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:16.923534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bc849-ea51-4fc6-bda2-60f0a112b9cc None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:16.947355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7240ae43-eafb-4d73-8d67-b58414f2e1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:16.949652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ae43-eafb-4d73-8d67-b58414f2e1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.950648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ae43-eafb-4d73-8d67-b58414f2e1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.951526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7240ae43-eafb-4d73-8d67-b58414f2e1f5 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:16.955911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240ae43-eafb-4d73-8d67-b58414f2e1f5 None admin] RBAC: Adding query filter params (name=network) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:16.958090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240ae43-eafb-4d73-8d67-b58414f2e1f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:16.967285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7240ae43-eafb-4d73-8d67-b58414f2e1f5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:16.988731 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 385/777] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:16 2022] GET /identity/v3/services?name=network => generated 124 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:16.990685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4fb02-d5cf-45c4-b044-82b9baa1885e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:16.992859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4fb02-d5cf-45c4-b044-82b9baa1885e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:16.993355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4fb02-d5cf-45c4-b044-82b9baa1885e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:17.030220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f2f4fb02-d5cf-45c4-b044-82b9baa1885e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:17.032561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f2f4fb02-d5cf-45c4-b044-82b9baa1885e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:17.072677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:17.167196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184bc849-ea51-4fc6-bda2-60f0a112b9cc None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:17.217356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0a97f-9ca3-46a0-9d67-f19cfffa8796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:17.218654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0a97f-9ca3-46a0-9d67-f19cfffa8796 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:17.219229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0a97f-9ca3-46a0-9d67-f19cfffa8796 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:17.219710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0a97f-9ca3-46a0-9d67-f19cfffa8796 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:17.220679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0a97f-9ca3-46a0-9d67-f19cfffa8796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:17.223125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0a97f-9ca3-46a0-9d67-f19cfffa8796 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:17.236865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 386/778] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:16 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:17.236865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4fb02-d5cf-45c4-b044-82b9baa1885e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:17.238282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 393/779] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Sep 6 22:30:16 2022] GET /identity/v3/services => generated 772 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:17.377134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:17.443741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4fb02-d5cf-45c4-b044-82b9baa1885e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:17.491257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29f57e-cb25-48d0-9397-b01db012d61b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:17.491807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29f57e-cb25-48d0-9397-b01db012d61b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:17.492088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29f57e-cb25-48d0-9397-b01db012d61b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:17.492359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b29f57e-cb25-48d0-9397-b01db012d61b None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:17.494553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29f57e-cb25-48d0-9397-b01db012d61b None admin] RBAC: Adding query filter params (type=network) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:17.494866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29f57e-cb25-48d0-9397-b01db012d61b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:17.496238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b29f57e-cb25-48d0-9397-b01db012d61b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:17.502847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 387/780] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:17 2022] GET /identity/v3/services?type=network => generated 350 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:17.509750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0a97f-9ca3-46a0-9d67-f19cfffa8796 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:17.692541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29f57e-cb25-48d0-9397-b01db012d61b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:17.702749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0a97f-9ca3-46a0-9d67-f19cfffa8796 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:17.738806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:17.839410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d704ff-1137-4116-b2a4-d007b716e034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:17.839958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d704ff-1137-4116-b2a4-d007b716e034 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:17.840236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d704ff-1137-4116-b2a4-d007b716e034 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:17.840510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d704ff-1137-4116-b2a4-d007b716e034 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:17.849341 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d704ff-1137-4116-b2a4-d007b716e034 None admin] RBAC: Adding query filter params (interface=public, service_id=88124d09915b44b1ba20ff551bf45fec, region_id=RegionOne) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:17.849661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d704ff-1137-4116-b2a4-d007b716e034 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:17.851034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d704ff-1137-4116-b2a4-d007b716e034 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:17.874831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 394/781] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Sep 6 22:30:17 2022] GET /identity/v3/endpoints?service_id=88124d09915b44b1ba20ff551bf45fec&interface=public®ion_id=RegionOne => generated 194 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:17.880106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b29f57e-cb25-48d0-9397-b01db012d61b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:17.881224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d704ff-1137-4116-b2a4-d007b716e034 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:17.972822 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:17.985683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdc925-ce21-4a2a-80e6-762c0695ef51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:17.986308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdc925-ce21-4a2a-80e6-762c0695ef51 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:17.986584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdc925-ce21-4a2a-80e6-762c0695ef51 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:17.986852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfdc925-ce21-4a2a-80e6-762c0695ef51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:18.019303 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfdc925-ce21-4a2a-80e6-762c0695ef51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-dbfdc925-ce21-4a2a-80e6-762c0695ef51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:18.027383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:18.028325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:18.029652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 388/782] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:30:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:18.043968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdc925-ce21-4a2a-80e6-762c0695ef51 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:18.144828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d704ff-1137-4116-b2a4-d007b716e034 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:18.219518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2bd4b-e94c-4654-9b3e-559b8d94baf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:18.220780 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2bd4b-e94c-4654-9b3e-559b8d94baf3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:18.220780 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2bd4b-e94c-4654-9b3e-559b8d94baf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:18.220780 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e2bd4b-e94c-4654-9b3e-559b8d94baf3 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:18.221264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2bd4b-e94c-4654-9b3e-559b8d94baf3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:18.224779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e2bd4b-e94c-4654-9b3e-559b8d94baf3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:18.238568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 395/783] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Sep 6 22:30:17 2022] GET /identity/v3/services => generated 772 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:18.248409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfdc925-ce21-4a2a-80e6-762c0695ef51 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:18.298762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb92bf-eddf-490e-9bce-ea1210ddab58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:18.299441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb92bf-eddf-490e-9bce-ea1210ddab58 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:18.299881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb92bf-eddf-490e-9bce-ea1210ddab58 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:18.308784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb92bf-eddf-490e-9bce-ea1210ddab58 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:18.312102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb92bf-eddf-490e-9bce-ea1210ddab58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:18.312102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb92bf-eddf-490e-9bce-ea1210ddab58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:18.312468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb92bf-eddf-490e-9bce-ea1210ddab58 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:18.341804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 389/784] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:30:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:18.342953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2bd4b-e94c-4654-9b3e-559b8d94baf3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:18.451484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:18.546282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e2bd4b-e94c-4654-9b3e-559b8d94baf3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:18.594753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d08f1-0ec8-4312-9d88-2cd170ca84b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:18.595387 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d08f1-0ec8-4312-9d88-2cd170ca84b6 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:18.595965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d08f1-0ec8-4312-9d88-2cd170ca84b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:18.595965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0d08f1-0ec8-4312-9d88-2cd170ca84b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:18.608023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d08f1-0ec8-4312-9d88-2cd170ca84b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:18.609745 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0d08f1-0ec8-4312-9d88-2cd170ca84b6 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-4a0d08f1-0ec8-4312-9d88-2cd170ca84b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:18.635584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:18.636587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:18.637513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:18.637513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:18.637513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:18.637513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 396/785] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:18.646147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb92bf-eddf-490e-9bce-ea1210ddab58 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:18.770870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:18.857123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb92bf-eddf-490e-9bce-ea1210ddab58 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:18.940790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe9f21-54c1-4fbf-8494-98f975f8eca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:18.940790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe9f21-54c1-4fbf-8494-98f975f8eca9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:18.940790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe9f21-54c1-4fbf-8494-98f975f8eca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:18.940790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbe9f21-54c1-4fbf-8494-98f975f8eca9 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:18.942486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe9f21-54c1-4fbf-8494-98f975f8eca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:18.942486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe9f21-54c1-4fbf-8494-98f975f8eca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:18.942486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbe9f21-54c1-4fbf-8494-98f975f8eca9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:18.968785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 390/786] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:18.976413 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d08f1-0ec8-4312-9d88-2cd170ca84b6 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:19.104782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:19.134527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0d08f1-0ec8-4312-9d88-2cd170ca84b6 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:19.241692 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4921190f-0d1e-44d3-81ff-224eb10caaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:19.242360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921190f-0d1e-44d3-81ff-224eb10caaf8 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:19.248802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921190f-0d1e-44d3-81ff-224eb10caaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:19.248802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4921190f-0d1e-44d3-81ff-224eb10caaf8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:19.260788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921190f-0d1e-44d3-81ff-224eb10caaf8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:19.260788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4921190f-0d1e-44d3-81ff-224eb10caaf8 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-4921190f-0d1e-44d3-81ff-224eb10caaf8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:19.280786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:19.281804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:19.282763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:19.282763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:19.282763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:19.282763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:19.282763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:19.282763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:19.282763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:19.282763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:19.282763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:30:19.282763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:19.282763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 397/787] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Sep 6 22:30:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:19.289247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe9f21-54c1-4fbf-8494-98f975f8eca9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:19.376857 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a89679-d208-4ce6-be72-ebca87369aae None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:19.376857 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a89679-d208-4ce6-be72-ebca87369aae None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:19.376857 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a89679-d208-4ce6-be72-ebca87369aae None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:19.377430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 398/788] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:30:19.383300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732214-b565-4f32-a5fc-dd0aeb80b60b None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:19.388782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732214-b565-4f32-a5fc-dd0aeb80b60b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:19.388782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49732214-b565-4f32-a5fc-dd0aeb80b60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:19.437857 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-49732214-b565-4f32-a5fc-dd0aeb80b60b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:19.439061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-49732214-b565-4f32-a5fc-dd0aeb80b60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:19.459707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbe9f21-54c1-4fbf-8494-98f975f8eca9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:19.471912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:19.504786 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0526b17-dd1b-4685-9d46-f009b17e7098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:19.507474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0526b17-dd1b-4685-9d46-f009b17e7098 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:19.508184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0526b17-dd1b-4685-9d46-f009b17e7098 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:19.508591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0526b17-dd1b-4685-9d46-f009b17e7098 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:19.509566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0526b17-dd1b-4685-9d46-f009b17e7098 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:19.510014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0526b17-dd1b-4685-9d46-f009b17e7098 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:19.511650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0526b17-dd1b-4685-9d46-f009b17e7098 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:19.547738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 391/789] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Sep 6 22:30:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:19.560791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0526b17-dd1b-4685-9d46-f009b17e7098 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:19.652044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 399/790] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:19.680782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b11cc-fcaa-48d3-b6b7-151d0038e2ff None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:19.680782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b11cc-fcaa-48d3-b6b7-151d0038e2ff None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:19.680782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b11cc-fcaa-48d3-b6b7-151d0038e2ff None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:19.680782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 400/791] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:19.688781 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae46e9-faca-4cc7-8b71-405c3327a44a None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:19.688781 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae46e9-faca-4cc7-8b71-405c3327a44a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:19.688781 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ae46e9-faca-4cc7-8b71-405c3327a44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:19.724783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-73ae46e9-faca-4cc7-8b71-405c3327a44a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:19.728782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-73ae46e9-faca-4cc7-8b71-405c3327a44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:19.804787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0526b17-dd1b-4685-9d46-f009b17e7098 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:19.883501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307e25c-be6a-4612-973b-1f4c633f9785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:19.884117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307e25c-be6a-4612-973b-1f4c633f9785 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:19.884394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307e25c-be6a-4612-973b-1f4c633f9785 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:19.884662 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9307e25c-be6a-4612-973b-1f4c633f9785 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:19.892782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 401/792] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:19.937060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:19.942082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae46e9-faca-4cc7-8b71-405c3327a44a None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:19.964534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:19.994772 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307e25c-be6a-4612-973b-1f4c633f9785 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:19.999465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9307e25c-be6a-4612-973b-1f4c633f9785 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:20.171002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ae46e9-faca-4cc7-8b71-405c3327a44a None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:20.221337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69ed8c7-ccb0-4e3d-9fce-7dda923e84ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:20.221892 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed8c7-ccb0-4e3d-9fce-7dda923e84ae None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:20.222705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed8c7-ccb0-4e3d-9fce-7dda923e84ae None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:20.222970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69ed8c7-ccb0-4e3d-9fce-7dda923e84ae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:20.242418 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69ed8c7-ccb0-4e3d-9fce-7dda923e84ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-d69ed8c7-ccb0-4e3d-9fce-7dda923e84ae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:20.247914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:20.248846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:20.249697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:20.252285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 402/793] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:30:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:20.254738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61744d7a-7501-4313-b064-9177c803a266 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:20.255167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61744d7a-7501-4313-b064-9177c803a266 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:20.255508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61744d7a-7501-4313-b064-9177c803a266 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:20.257615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 403/794] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:20.258431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61744d7a-7501-4313-b064-9177c803a266 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:20.271036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 392/795] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:19 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 713 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:30:20.271814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307e25c-be6a-4612-973b-1f4c633f9785 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:20.349996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:20.387205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:20.416399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9307e25c-be6a-4612-973b-1f4c633f9785 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:20.422835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61744d7a-7501-4313-b064-9177c803a266 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:20.464541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8375c5-75a6-4367-9b46-da813caf7163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:20.465377 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8375c5-75a6-4367-9b46-da813caf7163 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:20.465661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8375c5-75a6-4367-9b46-da813caf7163 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:20.465939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8375c5-75a6-4367-9b46-da813caf7163 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:20.478611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8375c5-75a6-4367-9b46-da813caf7163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-2d8375c5-75a6-4367-9b46-da813caf7163 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:20.483732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:20.484884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:20.485923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:20.486697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343d4b8-640c-4984-bab2-af1bc78a3550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:20.487090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343d4b8-640c-4984-bab2-af1bc78a3550 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:20.487997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343d4b8-640c-4984-bab2-af1bc78a3550 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:20.488810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 404/796] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:30:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:20.493657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343d4b8-640c-4984-bab2-af1bc78a3550 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:20.493657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343d4b8-640c-4984-bab2-af1bc78a3550 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:20.493657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343d4b8-640c-4984-bab2-af1bc78a3550 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:20.495203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2ddce-b5ed-4038-99a6-2c20828260f1 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:20.495637 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9343d4b8-640c-4984-bab2-af1bc78a3550 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:20.495885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2ddce-b5ed-4038-99a6-2c20828260f1 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:20.496351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2ddce-b5ed-4038-99a6-2c20828260f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:20.516804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 393/797] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:30:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:20.517682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343d4b8-640c-4984-bab2-af1bc78a3550 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:20.527421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-36e2ddce-b5ed-4038-99a6-2c20828260f1 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:20.528906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-36e2ddce-b5ed-4038-99a6-2c20828260f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:20.651032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 405/798] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:20.651824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2ddce-b5ed-4038-99a6-2c20828260f1 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:20.688088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9343d4b8-640c-4984-bab2-af1bc78a3550 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:20.721893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:20.740874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b3df2-f6bb-4e62-a27c-ae799faece90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:20.740874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b3df2-f6bb-4e62-a27c-ae799faece90 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:20.740874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b3df2-f6bb-4e62-a27c-ae799faece90 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:20.740874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1b3df2-f6bb-4e62-a27c-ae799faece90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:20.751391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b3df2-f6bb-4e62-a27c-ae799faece90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:20.752966 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1b3df2-f6bb-4e62-a27c-ae799faece90 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-2a1b3df2-f6bb-4e62-a27c-ae799faece90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:20.770326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:20.771346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:20.772268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:20.772268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:20.772268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:20.776799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 394/799] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:20.776799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b3df2-f6bb-4e62-a27c-ae799faece90 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:20.877335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:20.897523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1b3df2-f6bb-4e62-a27c-ae799faece90 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:20.927007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2ddce-b5ed-4038-99a6-2c20828260f1 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:20.937821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146116db-dde6-4706-93c8-258e9e6a25e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:20.938448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146116db-dde6-4706-93c8-258e9e6a25e0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:20.938722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146116db-dde6-4706-93c8-258e9e6a25e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:20.938981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146116db-dde6-4706-93c8-258e9e6a25e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:20.951296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146116db-dde6-4706-93c8-258e9e6a25e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:20.952942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146116db-dde6-4706-93c8-258e9e6a25e0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:20.964195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd873aaa-b24c-4af7-828f-e289254c9434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:20.964813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd873aaa-b24c-4af7-828f-e289254c9434 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:20.965089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd873aaa-b24c-4af7-828f-e289254c9434 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:20.965360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd873aaa-b24c-4af7-828f-e289254c9434 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:20.966008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd873aaa-b24c-4af7-828f-e289254c9434 None admin] RBAC: Adding query filter params (name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:20.966299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd873aaa-b24c-4af7-828f-e289254c9434 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:20.967639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd873aaa-b24c-4af7-828f-e289254c9434 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-146116db-dde6-4706-93c8-258e9e6a25e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:20.968534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:20.969579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:20.970450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:20.970450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:20.970450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:20.970450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 395/800] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:20.970680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146116db-dde6-4706-93c8-258e9e6a25e0 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:20.980116 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 406/801] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:30:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:20.982241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e42744-0914-4293-b184-7910a5bdaa7f None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:20.982659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e42744-0914-4293-b184-7910a5bdaa7f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:20.983064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e42744-0914-4293-b184-7910a5bdaa7f None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:20.985134 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 407/802] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:20.986052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e42744-0914-4293-b184-7910a5bdaa7f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:21.015684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:21.082199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146116db-dde6-4706-93c8-258e9e6a25e0 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:21.098045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e42744-0914-4293-b184-7910a5bdaa7f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:21.118814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b4f2f-175a-4221-b8ea-83854236cb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:21.119418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b4f2f-175a-4221-b8ea-83854236cb91 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:21.119690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b4f2f-175a-4221-b8ea-83854236cb91 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:21.119956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343b4f2f-175a-4221-b8ea-83854236cb91 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:21.120583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b4f2f-175a-4221-b8ea-83854236cb91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:21.120905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b4f2f-175a-4221-b8ea-83854236cb91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:21.122153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:21.122265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343b4f2f-175a-4221-b8ea-83854236cb91 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:21.133729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd154a-b10c-4b11-b1c7-f06badffbb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:21.134350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd154a-b10c-4b11-b1c7-f06badffbb7b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:21.134628 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd154a-b10c-4b11-b1c7-f06badffbb7b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:21.134728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 396/803] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:21.134899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fd154a-b10c-4b11-b1c7-f06badffbb7b None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:21.135476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd154a-b10c-4b11-b1c7-f06badffbb7b None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:21.136855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fd154a-b10c-4b11-b1c7-f06badffbb7b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:21.137698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b4f2f-175a-4221-b8ea-83854236cb91 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-47fd154a-b10c-4b11-b1c7-f06badffbb7b None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:21.146021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:21.147013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:21.147850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 408/804] 10.222.0.39 () {56 vars in 1129 bytes} [Tue Sep 6 22:30:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:21.148645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd154a-b10c-4b11-b1c7-f06badffbb7b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:21.228867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:21.235869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343b4f2f-175a-4221-b8ea-83854236cb91 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:21.249358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fd154a-b10c-4b11-b1c7-f06badffbb7b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:21.270838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a5ab49-9e7c-47f5-b9dd-cee5536fba5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:21.271372 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5ab49-9e7c-47f5-b9dd-cee5536fba5b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:21.271644 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5ab49-9e7c-47f5-b9dd-cee5536fba5b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:21.271907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a5ab49-9e7c-47f5-b9dd-cee5536fba5b None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:21.272530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5ab49-9e7c-47f5-b9dd-cee5536fba5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:21.272851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5ab49-9e7c-47f5-b9dd-cee5536fba5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:21.274197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a5ab49-9e7c-47f5-b9dd-cee5536fba5b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:21.284677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35593785-8d5b-4baa-8ece-eaa7b05ee155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:21.285399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 397/805] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:21.285591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35593785-8d5b-4baa-8ece-eaa7b05ee155 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:21.285853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35593785-8d5b-4baa-8ece-eaa7b05ee155 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:21.286120 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35593785-8d5b-4baa-8ece-eaa7b05ee155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:21.286638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ec899-2aef-4720-af03-3ce4f25d7a4f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:21.286914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ec899-2aef-4720-af03-3ce4f25d7a4f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:21.287183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2ec899-2aef-4720-af03-3ce4f25d7a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:21.301721 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35593785-8d5b-4baa-8ece-eaa7b05ee155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:21.303184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35593785-8d5b-4baa-8ece-eaa7b05ee155 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:21.306509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-9a2ec899-2aef-4720-af03-3ce4f25d7a4f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:21.307300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9a2ec899-2aef-4720-af03-3ce4f25d7a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-35593785-8d5b-4baa-8ece-eaa7b05ee155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:21.321468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:21.322514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:21.323637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:21.323637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:21.323637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:21.323637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 409/806] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:21.324580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35593785-8d5b-4baa-8ece-eaa7b05ee155 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:21.369136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:21.377650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 398/807] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:21.379254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ec899-2aef-4720-af03-3ce4f25d7a4f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:21.428876 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35593785-8d5b-4baa-8ece-eaa7b05ee155 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:21.464746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab4390-c175-47fc-90fb-47ef029caa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:21.465390 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab4390-c175-47fc-90fb-47ef029caa07 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:21.465666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab4390-c175-47fc-90fb-47ef029caa07 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:21.465932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab4390-c175-47fc-90fb-47ef029caa07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:21.473305 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab4390-c175-47fc-90fb-47ef029caa07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:21.475544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab4390-c175-47fc-90fb-47ef029caa07 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-5cab4390-c175-47fc-90fb-47ef029caa07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:21.487014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:21.488481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:21.494543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:21.494543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:21.494543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:21.494543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:21.494543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:21.494543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:21.494543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:21.494543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:21.494543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 410/808] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:21.494543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab4390-c175-47fc-90fb-47ef029caa07 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:21.495118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:21.512891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2ec899-2aef-4720-af03-3ce4f25d7a4f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:21.552557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ebac2-8ed2-435e-95b3-bfc240493ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:21.553459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ebac2-8ed2-435e-95b3-bfc240493ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:21.553741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ebac2-8ed2-435e-95b3-bfc240493ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:21.554012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ebac2-8ed2-435e-95b3-bfc240493ce4 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:21.554642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ebac2-8ed2-435e-95b3-bfc240493ce4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:21.554942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ebac2-8ed2-435e-95b3-bfc240493ce4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:21.556746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241ebac2-8ed2-435e-95b3-bfc240493ce4 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:21.565963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 399/809] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Sep 6 22:30:21 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:21.565963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ebac2-8ed2-435e-95b3-bfc240493ce4 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:21.614749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:21.625063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab4390-c175-47fc-90fb-47ef029caa07 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:21.682752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01975b0-4aa9-4f03-a6a4-56633995a1d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:21.683373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01975b0-4aa9-4f03-a6a4-56633995a1d7 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:21.683648 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01975b0-4aa9-4f03-a6a4-56633995a1d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:21.683916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01975b0-4aa9-4f03-a6a4-56633995a1d7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:21.685040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ebac2-8ed2-435e-95b3-bfc240493ce4 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:21.692309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01975b0-4aa9-4f03-a6a4-56633995a1d7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:21.693751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e01975b0-4aa9-4f03-a6a4-56633995a1d7 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-e01975b0-4aa9-4f03-a6a4-56633995a1d7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:21.705036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:21.705976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 411/810] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:30:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:21.707661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01975b0-4aa9-4f03-a6a4-56633995a1d7 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:21.723046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3e30b-0c8b-4e23-a406-8b9cf9207323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:21.723659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3e30b-0c8b-4e23-a406-8b9cf9207323 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:21.723942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3e30b-0c8b-4e23-a406-8b9cf9207323 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:21.724214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b3e30b-0c8b-4e23-a406-8b9cf9207323 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:21.724886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3e30b-0c8b-4e23-a406-8b9cf9207323 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:21.725188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3e30b-0c8b-4e23-a406-8b9cf9207323 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:21.726541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b3e30b-0c8b-4e23-a406-8b9cf9207323 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:21.739271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 400/811] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:21.740076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3e30b-0c8b-4e23-a406-8b9cf9207323 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:21.751533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:21.811112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01975b0-4aa9-4f03-a6a4-56633995a1d7 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:21.840192 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b3e30b-0c8b-4e23-a406-8b9cf9207323 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:21.845667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef5886-40e4-41ad-9bf6-d7c40f87435e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:21.846266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef5886-40e4-41ad-9bf6-d7c40f87435e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:21.846536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef5886-40e4-41ad-9bf6-d7c40f87435e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:21.846798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ef5886-40e4-41ad-9bf6-d7c40f87435e None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:21.848123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef5886-40e4-41ad-9bf6-d7c40f87435e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:21.848421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef5886-40e4-41ad-9bf6-d7c40f87435e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:21.849873 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ef5886-40e4-41ad-9bf6-d7c40f87435e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:21.861452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 412/812] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:21.862216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef5886-40e4-41ad-9bf6-d7c40f87435e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:21.876348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9324f-db6d-46d7-b705-60caa62d019d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:21.876982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9324f-db6d-46d7-b705-60caa62d019d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:21.877260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9324f-db6d-46d7-b705-60caa62d019d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:21.877530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d9324f-db6d-46d7-b705-60caa62d019d None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:21.878149 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9324f-db6d-46d7-b705-60caa62d019d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:21.878682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9324f-db6d-46d7-b705-60caa62d019d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:21.880041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d9324f-db6d-46d7-b705-60caa62d019d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:21.884267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:21.884905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 401/813] 10.222.0.39 () {56 vars in 1124 bytes} [Tue Sep 6 22:30:21 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:21.885669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9324f-db6d-46d7-b705-60caa62d019d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:21.956565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:21.973428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ef5886-40e4-41ad-9bf6-d7c40f87435e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:21.986488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d9324f-db6d-46d7-b705-60caa62d019d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:22.008389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee31579-d83b-4c0d-be4a-31781f06913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.009040 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee31579-d83b-4c0d-be4a-31781f06913b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.009310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee31579-d83b-4c0d-be4a-31781f06913b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.009572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee31579-d83b-4c0d-be4a-31781f06913b None admin] PATH_INFO: `/v3/services/network` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.010141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee31579-d83b-4c0d-be4a-31781f06913b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.011476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee31579-d83b-4c0d-be4a-31781f06913b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-cee31579-d83b-4c0d-be4a-31781f06913b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:22.020006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:22.021031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 06 22:30:22.021867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:22.022595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7b3a9-8884-4290-b6a2-05a72b5365c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.023017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 413/814] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:21 2022] GET /identity/v3/services/network => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:22.023099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7b3a9-8884-4290-b6a2-05a72b5365c9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.023099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7b3a9-8884-4290-b6a2-05a72b5365c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.023244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee31579-d83b-4c0d-be4a-31781f06913b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:22.023312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d7b3a9-8884-4290-b6a2-05a72b5365c9 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.023312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7b3a9-8884-4290-b6a2-05a72b5365c9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:22.023312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7b3a9-8884-4290-b6a2-05a72b5365c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.024643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d7b3a9-8884-4290-b6a2-05a72b5365c9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.042152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 402/815] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Sep 6 22:30:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:22.042952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7b3a9-8884-4290-b6a2-05a72b5365c9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:22.114127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:22.130244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee31579-d83b-4c0d-be4a-31781f06913b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:22.140944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d7b3a9-8884-4290-b6a2-05a72b5365c9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:22.164081 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03143d-89d4-4054-b127-fd51122c65e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.164819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03143d-89d4-4054-b127-fd51122c65e1 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.165187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03143d-89d4-4054-b127-fd51122c65e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.165462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec03143d-89d4-4054-b127-fd51122c65e1 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.170776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:22.173877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03143d-89d4-4054-b127-fd51122c65e1 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.175336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec03143d-89d4-4054-b127-fd51122c65e1 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.177211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423da97-70d4-4dd6-975f-c61d379d9c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.177845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423da97-70d4-4dd6-975f-c61d379d9c49 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.178118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423da97-70d4-4dd6-975f-c61d379d9c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.178388 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423da97-70d4-4dd6-975f-c61d379d9c49 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.179096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423da97-70d4-4dd6-975f-c61d379d9c49 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=b16947313dad4d59a97cc8b016ee54c1) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:22.179402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423da97-70d4-4dd6-975f-c61d379d9c49 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.180833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423da97-70d4-4dd6-975f-c61d379d9c49 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.186407 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 403/816] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:22 2022] GET /identity/v3/role_assignments?user.id=b16947313dad4d59a97cc8b016ee54c1&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-ec03143d-89d4-4054-b127-fd51122c65e1 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:22.186525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:22.192528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:22.193757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:22.193757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:22.193757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:22.193757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:22.193757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:22.193757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:22.193757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:22.193757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:22.193757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:22.193757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:22.194302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423da97-70d4-4dd6-975f-c61d379d9c49 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:22.194378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 414/817] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Sep 6 22:30:22 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:22.194378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03143d-89d4-4054-b127-fd51122c65e1 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:22.312267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec03143d-89d4-4054-b127-fd51122c65e1 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:22.323899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423da97-70d4-4dd6-975f-c61d379d9c49 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:22.359367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75a812-3144-4521-811b-071e08a15445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.359976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75a812-3144-4521-811b-071e08a15445 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.360248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75a812-3144-4521-811b-071e08a15445 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.360513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75a812-3144-4521-811b-071e08a15445 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.361170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75a812-3144-4521-811b-071e08a15445 None admin] RBAC: Adding query filter params (name=network) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:22.361468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75a812-3144-4521-811b-071e08a15445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.362032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e649e6-e3c8-40ff-92d0-3c2e4f5be511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.363280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e649e6-e3c8-40ff-92d0-3c2e4f5be511 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.363476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75a812-3144-4521-811b-071e08a15445 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.363659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e649e6-e3c8-40ff-92d0-3c2e4f5be511 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.363906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e649e6-e3c8-40ff-92d0-3c2e4f5be511 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.364622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e649e6-e3c8-40ff-92d0-3c2e4f5be511 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.365989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e649e6-e3c8-40ff-92d0-3c2e4f5be511 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.368814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 415/818] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:22 2022] GET /identity/v3/services?name=network => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:22.369455 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75a812-3144-4521-811b-071e08a15445 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:22.386828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:22.395258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 404/819] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:30:22 2022] POST /identity/v3/endpoints => generated 346 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:22.396590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e649e6-e3c8-40ff-92d0-3c2e4f5be511 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:22.469008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:22.524929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75a812-3144-4521-811b-071e08a15445 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:22.556037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e649e6-e3c8-40ff-92d0-3c2e4f5be511 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:22.569397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f4f3b-665f-4226-96cc-644a6a3bb6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.570351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f4f3b-665f-4226-96cc-644a6a3bb6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.570791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f4f3b-665f-4226-96cc-644a6a3bb6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.571064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5f4f3b-665f-4226-96cc-644a6a3bb6e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.584466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f4f3b-665f-4226-96cc-644a6a3bb6e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.586455 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5f4f3b-665f-4226-96cc-644a6a3bb6e2 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-fc5f4f3b-665f-4226-96cc-644a6a3bb6e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:22.603262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:22.604281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:22.605217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:22.605217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:22.605217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:22.605217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 416/820] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:22.605971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f4f3b-665f-4226-96cc-644a6a3bb6e2 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:22.618404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bff02-7b8e-4c13-8dd4-0beb02b31d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.618956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bff02-7b8e-4c13-8dd4-0beb02b31d6c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.619230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bff02-7b8e-4c13-8dd4-0beb02b31d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.619501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46bff02-7b8e-4c13-8dd4-0beb02b31d6c None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.620234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46bff02-7b8e-4c13-8dd4-0beb02b31d6c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:22.620527 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46bff02-7b8e-4c13-8dd4-0beb02b31d6c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.622904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46bff02-7b8e-4c13-8dd4-0beb02b31d6c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.642194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 405/821] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Sep 6 22:30:22 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:22.642920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bff02-7b8e-4c13-8dd4-0beb02b31d6c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:22.684626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:22.745859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5f4f3b-665f-4226-96cc-644a6a3bb6e2 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:22.771136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46bff02-7b8e-4c13-8dd4-0beb02b31d6c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:22.773573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:22.785447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128cdad-cac4-4a31-91a6-738dd10e75b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.786428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128cdad-cac4-4a31-91a6-738dd10e75b8 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.786905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128cdad-cac4-4a31-91a6-738dd10e75b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.787389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9128cdad-cac4-4a31-91a6-738dd10e75b8 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.788211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128cdad-cac4-4a31-91a6-738dd10e75b8 None admin] RBAC: Adding query filter params (type=network) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:22.788723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128cdad-cac4-4a31-91a6-738dd10e75b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.790304 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9128cdad-cac4-4a31-91a6-738dd10e75b8 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.803484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 417/822] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:22 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:22.804701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128cdad-cac4-4a31-91a6-738dd10e75b8 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:22.815712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d17e6-1ce8-4b77-acf7-88d695af0058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.816237 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d17e6-1ce8-4b77-acf7-88d695af0058 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.816517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d17e6-1ce8-4b77-acf7-88d695af0058 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.816843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510d17e6-1ce8-4b77-acf7-88d695af0058 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.817503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d17e6-1ce8-4b77-acf7-88d695af0058 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:22.817806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d17e6-1ce8-4b77-acf7-88d695af0058 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.819183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510d17e6-1ce8-4b77-acf7-88d695af0058 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.836374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 406/823] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:22.837384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d17e6-1ce8-4b77-acf7-88d695af0058 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:22.920972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9128cdad-cac4-4a31-91a6-738dd10e75b8 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:22.949262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:22.949455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510d17e6-1ce8-4b77-acf7-88d695af0058 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:22.963442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31b633-4dec-4615-9a92-9cd4c2a8290e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.964471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31b633-4dec-4615-9a92-9cd4c2a8290e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.965048 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31b633-4dec-4615-9a92-9cd4c2a8290e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.965591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a31b633-4dec-4615-9a92-9cd4c2a8290e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.978606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31b633-4dec-4615-9a92-9cd4c2a8290e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.980551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a31b633-4dec-4615-9a92-9cd4c2a8290e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.988622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d180b-b280-42ce-9b81-ed65feb82178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:22.989327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d180b-b280-42ce-9b81-ed65feb82178 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:22.989955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d180b-b280-42ce-9b81-ed65feb82178 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:22.989955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d180b-b280-42ce-9b81-ed65feb82178 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:22.990514 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d180b-b280-42ce-9b81-ed65feb82178 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:22.992218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d180b-b280-42ce-9b81-ed65feb82178 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-5a31b633-4dec-4615-9a92-9cd4c2a8290e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:22.999611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:23.000537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:23.001505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:23.001505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:23.001505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:23.003933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 418/824] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:23.005101 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31b633-4dec-4615-9a92-9cd4c2a8290e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:23.024809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 407/825] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:30:22 2022] POST /identity/v3/endpoints => generated 337 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:23.024809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d180b-b280-42ce-9b81-ed65feb82178 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:23.079465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:23.239014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a31b633-4dec-4615-9a92-9cd4c2a8290e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:23.263834 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d180b-b280-42ce-9b81-ed65feb82178 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:23.291478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ae22c-35cb-423a-a195-9b3ea326050b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:23.292258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ae22c-35cb-423a-a195-9b3ea326050b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:23.292636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ae22c-35cb-423a-a195-9b3ea326050b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:23.293031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ae22c-35cb-423a-a195-9b3ea326050b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:23.301187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ae22c-35cb-423a-a195-9b3ea326050b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:23.302832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ae22c-35cb-423a-a195-9b3ea326050b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:23.307111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb98488-4359-452f-b0fd-6563084643e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:23.307886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb98488-4359-452f-b0fd-6563084643e6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:23.308303 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb98488-4359-452f-b0fd-6563084643e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:23.308712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb98488-4359-452f-b0fd-6563084643e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:23.309523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb98488-4359-452f-b0fd-6563084643e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:23.309972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb98488-4359-452f-b0fd-6563084643e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:23.311816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb98488-4359-452f-b0fd-6563084643e6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-4b0ae22c-35cb-423a-a195-9b3ea326050b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:23.316245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:23.317515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:23.318714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:23.318714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:23.318714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:23.318714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:23.318714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:23.318714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:23.318714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:23.318714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:23.318714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 419/826] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:23.323558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ae22c-35cb-423a-a195-9b3ea326050b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:23.334944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 408/827] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:23.337346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb98488-4359-452f-b0fd-6563084643e6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:23.366698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:23.446156 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:23.465253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ae22c-35cb-423a-a195-9b3ea326050b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:23.476789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb98488-4359-452f-b0fd-6563084643e6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:23.514559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ee348-7432-4bd1-b707-e602382db331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:23.515153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ee348-7432-4bd1-b707-e602382db331 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:23.515434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ee348-7432-4bd1-b707-e602382db331 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:23.515701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5ee348-7432-4bd1-b707-e602382db331 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:23.516487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ee348-7432-4bd1-b707-e602382db331 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:23.516838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ee348-7432-4bd1-b707-e602382db331 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:23.518268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5ee348-7432-4bd1-b707-e602382db331 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:23.526825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff188fe5-ef65-4662-883f-3e851cb98c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:23.527861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff188fe5-ef65-4662-883f-3e851cb98c8f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:23.528340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff188fe5-ef65-4662-883f-3e851cb98c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:23.528860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff188fe5-ef65-4662-883f-3e851cb98c8f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:23.532942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 420/828] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:23.538766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ee348-7432-4bd1-b707-e602382db331 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:23.540436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff188fe5-ef65-4662-883f-3e851cb98c8f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:23.542363 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff188fe5-ef65-4662-883f-3e851cb98c8f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-ff188fe5-ef65-4662-883f-3e851cb98c8f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:23.567438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:23.568503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:23.569473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:23.569473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:23.569473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:23.569473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:23.569473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:23.569473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:23.569473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:23.569473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:23.572159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 409/829] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:23.581275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff188fe5-ef65-4662-883f-3e851cb98c8f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:23.651244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:23.658251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5ee348-7432-4bd1-b707-e602382db331 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:23.695308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcda6bf-9e5c-4d15-a72c-610f5576150b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:23.695987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcda6bf-9e5c-4d15-a72c-610f5576150b None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:23.696229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcda6bf-9e5c-4d15-a72c-610f5576150b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:23.696499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfcda6bf-9e5c-4d15-a72c-610f5576150b None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/a3f142a9e141440cbc854ade4f4135a0/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:23.698550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcda6bf-9e5c-4d15-a72c-610f5576150b None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=a3f142a9e141440cbc854ade4f4135a0, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:23.700074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfcda6bf-9e5c-4d15-a72c-610f5576150b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:23.721082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff188fe5-ef65-4662-883f-3e851cb98c8f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:23.726041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 421/830] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:30:23 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/a3f142a9e141440cbc854ade4f4135a0/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:30:23.766814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcaaed-4ce8-42f4-9400-99257a585e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:23.767823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcaaed-4ce8-42f4-9400-99257a585e4a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:23.768322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcaaed-4ce8-42f4-9400-99257a585e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:23.768831 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcaaed-4ce8-42f4-9400-99257a585e4a None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:23.770803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcaaed-4ce8-42f4-9400-99257a585e4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:23.771358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcaaed-4ce8-42f4-9400-99257a585e4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:23.773140 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcaaed-4ce8-42f4-9400-99257a585e4a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:23.776451 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:23.793495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 410/831] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:23.799599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcda6bf-9e5c-4d15-a72c-610f5576150b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:23.914395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfcda6bf-9e5c-4d15-a72c-610f5576150b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:23.956977 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efb00eb-531d-4b5c-8187-25cb877c1112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:23.957894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb00eb-531d-4b5c-8187-25cb877c1112 None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:23.958260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb00eb-531d-4b5c-8187-25cb877c1112 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:23.958538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efb00eb-531d-4b5c-8187-25cb877c1112 None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/7da74f16cb9b4b15a902947103a25cf6/roles/29088ad68ec54d74bee2847e2272e4e0` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:23.973975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb00eb-531d-4b5c-8187-25cb877c1112 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=7da74f16cb9b4b15a902947103a25cf6, role_id=29088ad68ec54d74bee2847e2272e4e0)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:23.975509 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efb00eb-531d-4b5c-8187-25cb877c1112 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:23.989659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 422/832] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:30:23 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/7da74f16cb9b4b15a902947103a25cf6/roles/29088ad68ec54d74bee2847e2272e4e0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:30:24.572042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7437-c520-46de-be79-ae9508b3fe37 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:24.572594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7437-c520-46de-be79-ae9508b3fe37 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:24.572594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc7437-c520-46de-be79-ae9508b3fe37 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:24.574609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 411/833] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:24.582766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00ea55-12ef-42f6-ad99-27277623ef42 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:24.582766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00ea55-12ef-42f6-ad99-27277623ef42 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:24.583392 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c00ea55-12ef-42f6-ad99-27277623ef42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:24.595877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:24.603723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2c00ea55-12ef-42f6-ad99-27277623ef42 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:24.605384 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2c00ea55-12ef-42f6-ad99-27277623ef42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:24.687003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 423/834] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:24.721781 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deb1b3-168c-4452-a11a-59dc2da5f0e2 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:24.722613 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deb1b3-168c-4452-a11a-59dc2da5f0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:24.722943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21deb1b3-168c-4452-a11a-59dc2da5f0e2 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:24.724750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 412/835] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:24.730293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4422-e1a0-451b-a66f-17cd3ffe24a4 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:24.730576 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4422-e1a0-451b-a66f-17cd3ffe24a4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:24.730847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd4422-e1a0-451b-a66f-17cd3ffe24a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:24.750738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-09cd4422-e1a0-451b-a66f-17cd3ffe24a4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:24.751898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-09cd4422-e1a0-451b-a66f-17cd3ffe24a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:24.829640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 424/836] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:24 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:24.986506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21deb1b3-168c-4452-a11a-59dc2da5f0e2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:25.096162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21deb1b3-168c-4452-a11a-59dc2da5f0e2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:25.121019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4422-e1a0-451b-a66f-17cd3ffe24a4 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:25.138256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71eeb8-c5d2-4c4d-94f1-17a6147d1208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:25.139173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71eeb8-c5d2-4c4d-94f1-17a6147d1208 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:25.139450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71eeb8-c5d2-4c4d-94f1-17a6147d1208 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:25.139727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d71eeb8-c5d2-4c4d-94f1-17a6147d1208 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:25.140481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71eeb8-c5d2-4c4d-94f1-17a6147d1208 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:25.142195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d71eeb8-c5d2-4c4d-94f1-17a6147d1208 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-0d71eeb8-c5d2-4c4d-94f1-17a6147d1208 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:25.154700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:25.155643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:25.156536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 413/837] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Sep 6 22:30:24 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:25.165117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71eeb8-c5d2-4c4d-94f1-17a6147d1208 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:25.193141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:25.233023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:25.240241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd4422-e1a0-451b-a66f-17cd3ffe24a4 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:25.289401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479a87c-8f65-48f7-983e-30f333ed7fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:25.290117 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479a87c-8f65-48f7-983e-30f333ed7fcf None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:25.290324 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d71eeb8-c5d2-4c4d-94f1-17a6147d1208 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:25.290426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479a87c-8f65-48f7-983e-30f333ed7fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:25.290653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0479a87c-8f65-48f7-983e-30f333ed7fcf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:25.291237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479a87c-8f65-48f7-983e-30f333ed7fcf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:25.293804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0479a87c-8f65-48f7-983e-30f333ed7fcf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-0479a87c-8f65-48f7-983e-30f333ed7fcf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:25.305758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:25.306751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 22:30:25.307589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:25.308878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 425/838] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:30:25 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:25.313043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479a87c-8f65-48f7-983e-30f333ed7fcf None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:25.334761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cca72b-262b-4e8b-9243-93f497879391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:25.335460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cca72b-262b-4e8b-9243-93f497879391 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:25.335829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cca72b-262b-4e8b-9243-93f497879391 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:25.336169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cca72b-262b-4e8b-9243-93f497879391 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:25.336915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cca72b-262b-4e8b-9243-93f497879391 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:25.337284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cca72b-262b-4e8b-9243-93f497879391 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:25.338906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cca72b-262b-4e8b-9243-93f497879391 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:25.344007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 414/839] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Sep 6 22:30:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:25.355832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cca72b-262b-4e8b-9243-93f497879391 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:25.442116 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0479a87c-8f65-48f7-983e-30f333ed7fcf None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:25.487288 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:25.490717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cca72b-262b-4e8b-9243-93f497879391 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:25.496031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bc594-f822-4d59-84ae-c28749cdb59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:25.496852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bc594-f822-4d59-84ae-c28749cdb59f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:25.497282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bc594-f822-4d59-84ae-c28749cdb59f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:25.497711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4bc594-f822-4d59-84ae-c28749cdb59f None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:25.498486 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bc594-f822-4d59-84ae-c28749cdb59f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:25.498932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bc594-f822-4d59-84ae-c28749cdb59f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:25.500897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4bc594-f822-4d59-84ae-c28749cdb59f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:25.513790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 426/840] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:30:25 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:25.515143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bc594-f822-4d59-84ae-c28749cdb59f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:25.526341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:25.541253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7c75b6-a696-4c64-ab05-3845a4b33b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:25.541875 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c75b6-a696-4c64-ab05-3845a4b33b49 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:25.542212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c75b6-a696-4c64-ab05-3845a4b33b49 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:25.542535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7c75b6-a696-4c64-ab05-3845a4b33b49 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:25.543173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c75b6-a696-4c64-ab05-3845a4b33b49 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:25.543917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c75b6-a696-4c64-ab05-3845a4b33b49 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:25.545687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7c75b6-a696-4c64-ab05-3845a4b33b49 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:25.552837 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 415/841] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Sep 6 22:30:25 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:25.650317 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4bc594-f822-4d59-84ae-c28749cdb59f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:25.692015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36acc36-a2d8-4f47-917c-5e6ac653d23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:25.692633 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36acc36-a2d8-4f47-917c-5e6ac653d23d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:25.692965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36acc36-a2d8-4f47-917c-5e6ac653d23d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:25.693237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36acc36-a2d8-4f47-917c-5e6ac653d23d None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:25.693960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36acc36-a2d8-4f47-917c-5e6ac653d23d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:25.694269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36acc36-a2d8-4f47-917c-5e6ac653d23d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:25.695673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36acc36-a2d8-4f47-917c-5e6ac653d23d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:25.701525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 427/842] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:30:25 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:26.048266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb811d-51d7-4a6c-9642-6e6d5e2bea71 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:26.048798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb811d-51d7-4a6c-9642-6e6d5e2bea71 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:26.048798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb811d-51d7-4a6c-9642-6e6d5e2bea71 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:26.050767 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 416/843] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:26.056397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fedab-7794-4b22-9293-061f7b7f5668 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:26.056688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fedab-7794-4b22-9293-061f7b7f5668 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:26.057006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43fedab-7794-4b22-9293-061f7b7f5668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:26.076660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-f43fedab-7794-4b22-9293-061f7b7f5668 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:26.077894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f43fedab-7794-4b22-9293-061f7b7f5668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:26.136009 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:26.152350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 428/844] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:26.296507 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc872928-2465-49ef-a105-d115d1a73b10 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:26.296879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc872928-2465-49ef-a105-d115d1a73b10 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:26.297086 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc872928-2465-49ef-a105-d115d1a73b10 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:26.298607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 417/845] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:26.304664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3a01a-9cea-4ec5-9727-80f6ea0e4ffe None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:26.305209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3a01a-9cea-4ec5-9727-80f6ea0e4ffe None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:26.305688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3a01a-9cea-4ec5-9727-80f6ea0e4ffe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:26.325204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-26c3a01a-9cea-4ec5-9727-80f6ea0e4ffe None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:26.326657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-26c3a01a-9cea-4ec5-9727-80f6ea0e4ffe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:26.401223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 429/846] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:26 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:26.429875 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc872928-2465-49ef-a105-d115d1a73b10 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:26.540105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc872928-2465-49ef-a105-d115d1a73b10 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:26.580861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfed8e4-6c31-45b8-a677-5eff9387da52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:26.580861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfed8e4-6c31-45b8-a677-5eff9387da52 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:26.580861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfed8e4-6c31-45b8-a677-5eff9387da52 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:26.580861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfed8e4-6c31-45b8-a677-5eff9387da52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:26.586779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfed8e4-6c31-45b8-a677-5eff9387da52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-9cfed8e4-6c31-45b8-a677-5eff9387da52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:26.591817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:26.592823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:26.593935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 418/847] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:30:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:26.601505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3a01a-9cea-4ec5-9727-80f6ea0e4ffe None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:26.684019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfed8e4-6c31-45b8-a677-5eff9387da52 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:26.713487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3a01a-9cea-4ec5-9727-80f6ea0e4ffe None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:26.736819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:26.744062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:26.761237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc37202-dff0-4e88-ac0a-c232c7176e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:26.761941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc37202-dff0-4e88-ac0a-c232c7176e96 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:26.762277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc37202-dff0-4e88-ac0a-c232c7176e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:26.762637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc37202-dff0-4e88-ac0a-c232c7176e96 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:26.763366 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc37202-dff0-4e88-ac0a-c232c7176e96 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:26.763978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc37202-dff0-4e88-ac0a-c232c7176e96 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:26.766045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc37202-dff0-4e88-ac0a-c232c7176e96 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:26.782706 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 430/848] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:30:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:26.785400 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc37202-dff0-4e88-ac0a-c232c7176e96 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:26.810123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfed8e4-6c31-45b8-a677-5eff9387da52 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:26.854590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b8266-051a-4d17-970d-daf5d6178b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:26.855258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b8266-051a-4d17-970d-daf5d6178b90 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:26.855539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b8266-051a-4d17-970d-daf5d6178b90 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:26.855813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b8266-051a-4d17-970d-daf5d6178b90 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:26.871425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6b8266-051a-4d17-970d-daf5d6178b90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-7a6b8266-051a-4d17-970d-daf5d6178b90 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:26.879380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:26.880302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:26.881854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 419/849] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:30:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:26.886476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b8266-051a-4d17-970d-daf5d6178b90 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:26.920193 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc37202-dff0-4e88-ac0a-c232c7176e96 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:26.962584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f580f4-f993-40ae-9688-171b54485ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:26.963210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f580f4-f993-40ae-9688-171b54485ebe None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:26.963646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f580f4-f993-40ae-9688-171b54485ebe None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:26.963921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f580f4-f993-40ae-9688-171b54485ebe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:26.975260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f580f4-f993-40ae-9688-171b54485ebe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:26.976821 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f580f4-f993-40ae-9688-171b54485ebe None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-45f580f4-f993-40ae-9688-171b54485ebe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:26.991941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:26.993224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:26.994153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:26.994153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:26.994153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:26.997837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 431/850] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:27.000179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f580f4-f993-40ae-9688-171b54485ebe None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:27.018508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6b8266-051a-4d17-970d-daf5d6178b90 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:27.036038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:27.057982 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:27.067832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecf291-ead0-4eac-b6b8-7ddb1468abcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:27.069069 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecf291-ead0-4eac-b6b8-7ddb1468abcf None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:27.069515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecf291-ead0-4eac-b6b8-7ddb1468abcf None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:27.069935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ecf291-ead0-4eac-b6b8-7ddb1468abcf None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:27.070838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecf291-ead0-4eac-b6b8-7ddb1468abcf None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:27.071314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecf291-ead0-4eac-b6b8-7ddb1468abcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:27.073060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ecf291-ead0-4eac-b6b8-7ddb1468abcf None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:27.082952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 420/851] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:30:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:27.096775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecf291-ead0-4eac-b6b8-7ddb1468abcf None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:27.156860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f580f4-f993-40ae-9688-171b54485ebe None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:27.203198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6edc8-6e14-4bf9-a15c-54cfa4e5b4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:27.204213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6edc8-6e14-4bf9-a15c-54cfa4e5b4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:27.204498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6edc8-6e14-4bf9-a15c-54cfa4e5b4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:27.204884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6edc8-6e14-4bf9-a15c-54cfa4e5b4c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:27.205591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6edc8-6e14-4bf9-a15c-54cfa4e5b4c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:27.205898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6edc8-6e14-4bf9-a15c-54cfa4e5b4c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:27.207766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6edc8-6e14-4bf9-a15c-54cfa4e5b4c5 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:27.218362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ecf291-ead0-4eac-b6b8-7ddb1468abcf None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:27.220017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 432/852] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:27.226160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6edc8-6e14-4bf9-a15c-54cfa4e5b4c5 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:27.270127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad1fe7-31e2-4bf5-af6f-851ee4f8f717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:27.271398 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad1fe7-31e2-4bf5-af6f-851ee4f8f717 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:27.272891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad1fe7-31e2-4bf5-af6f-851ee4f8f717 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:27.273380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ad1fe7-31e2-4bf5-af6f-851ee4f8f717 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:27.289373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad1fe7-31e2-4bf5-af6f-851ee4f8f717 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:27.290820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ad1fe7-31e2-4bf5-af6f-851ee4f8f717 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-a7ad1fe7-31e2-4bf5-af6f-851ee4f8f717 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:27.309239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:27.310230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:27.311178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:27.311178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:27.311178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:27.315560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 421/853] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:27.316136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad1fe7-31e2-4bf5-af6f-851ee4f8f717 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:27.342565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:27.351872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:27.371705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6edc8-6e14-4bf9-a15c-54cfa4e5b4c5 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:27.409248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270bb00-3988-47be-a38e-d542ff5cea02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:27.410370 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270bb00-3988-47be-a38e-d542ff5cea02 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:27.410829 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270bb00-3988-47be-a38e-d542ff5cea02 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:27.411397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270bb00-3988-47be-a38e-d542ff5cea02 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:27.419488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270bb00-3988-47be-a38e-d542ff5cea02 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:27.421393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270bb00-3988-47be-a38e-d542ff5cea02 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:27.430886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ad1fe7-31e2-4bf5-af6f-851ee4f8f717 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-1270bb00-3988-47be-a38e-d542ff5cea02 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:27.434837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:27.435803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:27.436750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:27.436750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:27.436750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:27.436750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:27.436750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:27.436750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:27.436750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:27.436750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:27.436750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 06 22:30:27.436750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:27.444716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 433/854] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:30:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:27.450295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270bb00-3988-47be-a38e-d542ff5cea02 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:27.473095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf233d3-9e51-4138-b116-a10d676ed50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:27.475195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf233d3-9e51-4138-b116-a10d676ed50d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:27.475474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf233d3-9e51-4138-b116-a10d676ed50d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:27.475746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf233d3-9e51-4138-b116-a10d676ed50d None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:27.476399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf233d3-9e51-4138-b116-a10d676ed50d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:27.476699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf233d3-9e51-4138-b116-a10d676ed50d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:27.478090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf233d3-9e51-4138-b116-a10d676ed50d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:27.499281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 422/855] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:27.500185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf233d3-9e51-4138-b116-a10d676ed50d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:27.601877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270bb00-3988-47be-a38e-d542ff5cea02 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:27.608490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:27.631162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf233d3-9e51-4138-b116-a10d676ed50d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:27.645414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d569c-92a3-41f1-bd3a-4ca574d98da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:27.646411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d569c-92a3-41f1-bd3a-4ca574d98da9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:27.647213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d569c-92a3-41f1-bd3a-4ca574d98da9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:27.647798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1d569c-92a3-41f1-bd3a-4ca574d98da9 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:27.648818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d569c-92a3-41f1-bd3a-4ca574d98da9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:27.649445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d569c-92a3-41f1-bd3a-4ca574d98da9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:27.651326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1d569c-92a3-41f1-bd3a-4ca574d98da9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:27.668747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d59459-379e-4627-a62f-7dc7a64e9caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:27.669842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d59459-379e-4627-a62f-7dc7a64e9caa None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:27.669842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d59459-379e-4627-a62f-7dc7a64e9caa None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:27.670173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d59459-379e-4627-a62f-7dc7a64e9caa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:27.679871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 434/856] 10.222.0.39 () {56 vars in 1135 bytes} [Tue Sep 6 22:30:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:27.681522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d59459-379e-4627-a62f-7dc7a64e9caa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:27.683746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d59459-379e-4627-a62f-7dc7a64e9caa None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:27.685307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d569c-92a3-41f1-bd3a-4ca574d98da9 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-d3d59459-379e-4627-a62f-7dc7a64e9caa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:27.700525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:27.701566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:27.702499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:27.702499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:27.702499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:27.702499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:27.702499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:27.702499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:27.702499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:27.702499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:27.702499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:27.702499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:27.703607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:27.707627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 423/857] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Sep 6 22:30:27 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:27.708656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d59459-379e-4627-a62f-7dc7a64e9caa None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:27.815316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1d569c-92a3-41f1-bd3a-4ca574d98da9 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:27.827057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d59459-379e-4627-a62f-7dc7a64e9caa None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:27.855644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679015cc-fa67-4a09-b87b-411f2341f341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:27.856700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679015cc-fa67-4a09-b87b-411f2341f341 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:27.857224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679015cc-fa67-4a09-b87b-411f2341f341 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:27.857702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679015cc-fa67-4a09-b87b-411f2341f341 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:27.869713 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb4c71d-129a-4725-b410-e1680525ce71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:27.870342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4c71d-129a-4725-b410-e1680525ce71 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:27.870618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4c71d-129a-4725-b410-e1680525ce71 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:27.870888 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb4c71d-129a-4725-b410-e1680525ce71 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:27.872021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4c71d-129a-4725-b410-e1680525ce71 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:27.872021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4c71d-129a-4725-b410-e1680525ce71 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:27.873839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb4c71d-129a-4725-b410-e1680525ce71 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:27.890498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:27.898905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 424/858] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Sep 6 22:30:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:27.900300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddae8b-3a2b-4ea1-977b-3136675ff61e None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:27.900582 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddae8b-3a2b-4ea1-977b-3136675ff61e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:27.900883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbddae8b-3a2b-4ea1-977b-3136675ff61e None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:27.902260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 425/859] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:27.907307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddae8b-3a2b-4ea1-977b-3136675ff61e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:27.941612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679015cc-fa67-4a09-b87b-411f2341f341 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:27.942851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679015cc-fa67-4a09-b87b-411f2341f341 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:27.954220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:28.016915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbddae8b-3a2b-4ea1-977b-3136675ff61e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:28.044662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 435/860] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 22:30:28.045937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e023-4efe-46e5-99b4-f9bd09924c08 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:28.046221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e023-4efe-46e5-99b4-f9bd09924c08 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:28.046490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807e023-4efe-46e5-99b4-f9bd09924c08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:28.053433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de9da2b-915c-48e4-8a85-94a66458e240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:28.054036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9da2b-915c-48e4-8a85-94a66458e240 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:28.054339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9da2b-915c-48e4-8a85-94a66458e240 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:28.054611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de9da2b-915c-48e4-8a85-94a66458e240 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:28.064240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-9807e023-4efe-46e5-99b4-f9bd09924c08 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:28.065466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9807e023-4efe-46e5-99b4-f9bd09924c08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:28.120869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:28.124945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de9da2b-915c-48e4-8a85-94a66458e240 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:28.126668 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de9da2b-915c-48e4-8a85-94a66458e240 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:28.131859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 436/861] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:28.133519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a57af-d796-4f7c-8a3e-5e4135934764 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:28.133801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a57af-d796-4f7c-8a3e-5e4135934764 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:28.134067 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49a57af-d796-4f7c-8a3e-5e4135934764 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:28.135270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 437/862] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:28.135944 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a57af-d796-4f7c-8a3e-5e4135934764 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:28.185423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:28.248883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 426/863] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:27 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:30:28.248883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364842d-e1b7-401c-a27c-eb4806c4fe52 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:28.248883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364842d-e1b7-401c-a27c-eb4806c4fe52 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:28.248883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2364842d-e1b7-401c-a27c-eb4806c4fe52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:28.253092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49a57af-d796-4f7c-8a3e-5e4135934764 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:28.266074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2364842d-e1b7-401c-a27c-eb4806c4fe52 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:28.267706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2364842d-e1b7-401c-a27c-eb4806c4fe52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:28.289922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473c679-ce48-4ff1-8534-21569e3caa66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:28.290537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c679-ce48-4ff1-8534-21569e3caa66 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:28.290804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c679-ce48-4ff1-8534-21569e3caa66 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:28.291074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1473c679-ce48-4ff1-8534-21569e3caa66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:28.301779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473c679-ce48-4ff1-8534-21569e3caa66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:28.304634 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1473c679-ce48-4ff1-8534-21569e3caa66 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-1473c679-ce48-4ff1-8534-21569e3caa66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:28.321011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:28.322161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:28.323268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:28.323268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:28.323268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:28.324730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 438/864] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:28.325654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473c679-ce48-4ff1-8534-21569e3caa66 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:28.352866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 427/865] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:28 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:28.352866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364842d-e1b7-401c-a27c-eb4806c4fe52 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:28.395498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:28.432287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:28.449707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1473c679-ce48-4ff1-8534-21569e3caa66 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:28.470919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2364842d-e1b7-401c-a27c-eb4806c4fe52 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:28.487433 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c240db-4ccb-4349-ab8e-62d500063f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:28.488315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c240db-4ccb-4349-ab8e-62d500063f51 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:28.488584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c240db-4ccb-4349-ab8e-62d500063f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:28.488877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c240db-4ccb-4349-ab8e-62d500063f51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:28.499160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c240db-4ccb-4349-ab8e-62d500063f51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:28.500512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c240db-4ccb-4349-ab8e-62d500063f51 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:28.508917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9329e-dd6c-42d3-a594-2bc30ad9751a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:28.509522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9329e-dd6c-42d3-a594-2bc30ad9751a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:28.509808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9329e-dd6c-42d3-a594-2bc30ad9751a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:28.510072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b9329e-dd6c-42d3-a594-2bc30ad9751a None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:28.510715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9329e-dd6c-42d3-a594-2bc30ad9751a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:28.511016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9329e-dd6c-42d3-a594-2bc30ad9751a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:28.512372 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b9329e-dd6c-42d3-a594-2bc30ad9751a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-86c240db-4ccb-4349-ab8e-62d500063f51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:28.516925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:28.517841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:28.518759 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:28.518759 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:28.518759 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:28.520541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 439/866] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:28.521629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c240db-4ccb-4349-ab8e-62d500063f51 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:28.528368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 428/867] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:28.529442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9329e-dd6c-42d3-a594-2bc30ad9751a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:28.635830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:28.638554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c240db-4ccb-4349-ab8e-62d500063f51 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:28.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b9329e-dd6c-42d3-a594-2bc30ad9751a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:28.672479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3d7bf-9445-4687-bbff-002cc91dd570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:28.673200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3d7bf-9445-4687-bbff-002cc91dd570 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:28.673414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3d7bf-9445-4687-bbff-002cc91dd570 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:28.673681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3d7bf-9445-4687-bbff-002cc91dd570 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:28.674333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3d7bf-9445-4687-bbff-002cc91dd570 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:28.675687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b3d7bf-9445-4687-bbff-002cc91dd570 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:28.683774 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eac433-239f-42f7-8b55-406469943dbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:28.684367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eac433-239f-42f7-8b55-406469943dbe None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:28.684643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eac433-239f-42f7-8b55-406469943dbe None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:28.684948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5eac433-239f-42f7-8b55-406469943dbe None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:28.685581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eac433-239f-42f7-8b55-406469943dbe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:28.685883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eac433-239f-42f7-8b55-406469943dbe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:28.687370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5eac433-239f-42f7-8b55-406469943dbe None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:28.691461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 440/868] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:30:28 2022] POST /identity/v3/services => generated 269 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:28.692099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3d7bf-9445-4687-bbff-002cc91dd570 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:28.696273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:28.709067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 429/869] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:28.710259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eac433-239f-42f7-8b55-406469943dbe None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:28.824791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b3d7bf-9445-4687-bbff-002cc91dd570 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:28.828122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5eac433-239f-42f7-8b55-406469943dbe None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:28.861917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ce764-a9fc-4d69-b069-6aa39760ae8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:28.863878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ce764-a9fc-4d69-b069-6aa39760ae8a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:28.863878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ce764-a9fc-4d69-b069-6aa39760ae8a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:28.864101 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4ce764-a9fc-4d69-b069-6aa39760ae8a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:28.872063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ce764-a9fc-4d69-b069-6aa39760ae8a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:28.873408 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0810f-4628-488a-b52d-bd21b9bb1e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:28.873765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4ce764-a9fc-4d69-b069-6aa39760ae8a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:28.874159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0810f-4628-488a-b52d-bd21b9bb1e58 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:28.874508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0810f-4628-488a-b52d-bd21b9bb1e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:28.875072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b0810f-4628-488a-b52d-bd21b9bb1e58 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:28.875734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0810f-4628-488a-b52d-bd21b9bb1e58 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:28.877238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b0810f-4628-488a-b52d-bd21b9bb1e58 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-6e4ce764-a9fc-4d69-b069-6aa39760ae8a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:28.889546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:28.890651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:28.891784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:28.891784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:28.891784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:28.891784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:28.891784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:28.891784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:28.891784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:28.891784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:28.891784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 441/870] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:28.891784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ce764-a9fc-4d69-b069-6aa39760ae8a None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:28.896511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 430/871] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:30:28 2022] POST /identity/v3/services => generated 251 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:28.897427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0810f-4628-488a-b52d-bd21b9bb1e58 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:28.912071 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:28.984489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:29.035365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4ce764-a9fc-4d69-b069-6aa39760ae8a None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:29.063065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b0810f-4628-488a-b52d-bd21b9bb1e58 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:29.085348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622166f-d77b-40da-8637-1c16b801d450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:29.086496 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622166f-d77b-40da-8637-1c16b801d450 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:29.087293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622166f-d77b-40da-8637-1c16b801d450 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:29.087773 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9622166f-d77b-40da-8637-1c16b801d450 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:29.096440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622166f-d77b-40da-8637-1c16b801d450 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:29.098813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9622166f-d77b-40da-8637-1c16b801d450 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:29.116111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1831d-09e7-4bc3-b26f-1ca02dc45ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-9622166f-d77b-40da-8637-1c16b801d450 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:29.116372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:29.117437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:29.118459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:29.118459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:29.118459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:29.118459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:29.118459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:29.118459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:29.118459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:29.118459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:29.118459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 442/872] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:29.119683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1831d-09e7-4bc3-b26f-1ca02dc45ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:29.119964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1831d-09e7-4bc3-b26f-1ca02dc45ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:29.120323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1831d-09e7-4bc3-b26f-1ca02dc45ef5 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:29.121774 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1831d-09e7-4bc3-b26f-1ca02dc45ef5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:29.121774 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1831d-09e7-4bc3-b26f-1ca02dc45ef5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:29.123072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c1831d-09e7-4bc3-b26f-1ca02dc45ef5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:29.124207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622166f-d77b-40da-8637-1c16b801d450 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:29.136620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 431/873] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:29.144119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1831d-09e7-4bc3-b26f-1ca02dc45ef5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:29.242336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:29.246745 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9622166f-d77b-40da-8637-1c16b801d450 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:29.269150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:29.285889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63703f-1830-4c29-b6d0-68c69939e945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:29.286647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63703f-1830-4c29-b6d0-68c69939e945 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:29.287020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63703f-1830-4c29-b6d0-68c69939e945 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:29.287461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63703f-1830-4c29-b6d0-68c69939e945 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:29.288328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63703f-1830-4c29-b6d0-68c69939e945 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:29.288727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63703f-1830-4c29-b6d0-68c69939e945 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:29.290269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63703f-1830-4c29-b6d0-68c69939e945 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:29.293799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c1831d-09e7-4bc3-b26f-1ca02dc45ef5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:29.313139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 443/874] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:29.314764 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63703f-1830-4c29-b6d0-68c69939e945 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:29.357302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0eec48c-6593-4c72-a5fe-2877ffdfcce0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:29.358830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eec48c-6593-4c72-a5fe-2877ffdfcce0 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:29.359257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eec48c-6593-4c72-a5fe-2877ffdfcce0 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:29.359671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0eec48c-6593-4c72-a5fe-2877ffdfcce0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:29.361199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eec48c-6593-4c72-a5fe-2877ffdfcce0 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=a3f142a9e141440cbc854ade4f4135a0) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:29.361683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eec48c-6593-4c72-a5fe-2877ffdfcce0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:29.364696 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0eec48c-6593-4c72-a5fe-2877ffdfcce0 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:29.372645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 432/875] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:29 2022] GET /identity/v3/role_assignments?user.id=a3f142a9e141440cbc854ade4f4135a0&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:29.462529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63703f-1830-4c29-b6d0-68c69939e945 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:29.501925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6d47aa-7e55-439d-ac42-3e79400ddbca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:29.502538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d47aa-7e55-439d-ac42-3e79400ddbca None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:29.502812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d47aa-7e55-439d-ac42-3e79400ddbca None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:29.503080 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6d47aa-7e55-439d-ac42-3e79400ddbca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:29.504487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d47aa-7e55-439d-ac42-3e79400ddbca None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=7da74f16cb9b4b15a902947103a25cf6) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:29.504846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d47aa-7e55-439d-ac42-3e79400ddbca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:29.506231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6d47aa-7e55-439d-ac42-3e79400ddbca None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:29.513715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 444/876] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:29 2022] GET /identity/v3/role_assignments?user.id=7da74f16cb9b4b15a902947103a25cf6&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:30.988795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d678bd-a43f-4c97-9fbe-6e86781e5f09 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:30.988795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d678bd-a43f-4c97-9fbe-6e86781e5f09 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:30.988795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d678bd-a43f-4c97-9fbe-6e86781e5f09 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:30.989449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 433/877] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:30.994790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90f80e-0a99-4db5-8392-dcecf1bdae16 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:30.995180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90f80e-0a99-4db5-8392-dcecf1bdae16 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:30.995338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb90f80e-0a99-4db5-8392-dcecf1bdae16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:31.015453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-cb90f80e-0a99-4db5-8392-dcecf1bdae16 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:31.017089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-cb90f80e-0a99-4db5-8392-dcecf1bdae16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:31.060316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:31.091177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 445/878] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:30 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:31.198108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c54dc-981b-43f5-bf1a-ea0176323469 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:31.198461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c54dc-981b-43f5-bf1a-ea0176323469 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:31.198749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c54dc-981b-43f5-bf1a-ea0176323469 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:31.200339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 434/879] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:31.205416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75298fcb-52f7-432d-931d-bc0ea2e17087 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:31.205699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75298fcb-52f7-432d-931d-bc0ea2e17087 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:31.205972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75298fcb-52f7-432d-931d-bc0ea2e17087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:31.225495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-75298fcb-52f7-432d-931d-bc0ea2e17087 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:31.226704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-75298fcb-52f7-432d-931d-bc0ea2e17087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:31.296975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 446/880] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:31.374310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c54dc-981b-43f5-bf1a-ea0176323469 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:31.481876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c54dc-981b-43f5-bf1a-ea0176323469 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:31.512144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:31.517713 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379c413-746a-4466-b95b-dd787cba73e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:31.518698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379c413-746a-4466-b95b-dd787cba73e8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:31.519143 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379c413-746a-4466-b95b-dd787cba73e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:31.519438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1379c413-746a-4466-b95b-dd787cba73e8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:31.520081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379c413-746a-4466-b95b-dd787cba73e8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:31.521455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1379c413-746a-4466-b95b-dd787cba73e8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-1379c413-746a-4466-b95b-dd787cba73e8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:31.530730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:31.531743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 22:30:31.532625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:31.536623 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 435/881] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Sep 6 22:30:31 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:31.549904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75298fcb-52f7-432d-931d-bc0ea2e17087 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:31.582827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379c413-746a-4466-b95b-dd787cba73e8 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:31.668629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75298fcb-52f7-432d-931d-bc0ea2e17087 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:31.671367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:31.696266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1379c413-746a-4466-b95b-dd787cba73e8 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:31.707896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5d7949-ed80-4653-8757-bdbb282aa325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:31.708533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d7949-ed80-4653-8757-bdbb282aa325 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:31.708841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d7949-ed80-4653-8757-bdbb282aa325 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:31.709164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5d7949-ed80-4653-8757-bdbb282aa325 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:31.709764 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d7949-ed80-4653-8757-bdbb282aa325 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:31.710064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d7949-ed80-4653-8757-bdbb282aa325 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:31.711497 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5d7949-ed80-4653-8757-bdbb282aa325 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:31.716839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 447/882] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Sep 6 22:30:31 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:31.718145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e69f2d-0f1a-4deb-a87d-804c28c5f0eb None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:31.718395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e69f2d-0f1a-4deb-a87d-804c28c5f0eb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:31.718669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e69f2d-0f1a-4deb-a87d-804c28c5f0eb None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:31.720208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 448/883] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:31.725444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e69f2d-0f1a-4deb-a87d-804c28c5f0eb None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:31.743981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc398497-46a6-4962-86b7-a67b46f2249a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:31.744926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc398497-46a6-4962-86b7-a67b46f2249a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:31.745206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc398497-46a6-4962-86b7-a67b46f2249a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:31.745469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc398497-46a6-4962-86b7-a67b46f2249a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:31.746054 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc398497-46a6-4962-86b7-a67b46f2249a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:31.747404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc398497-46a6-4962-86b7-a67b46f2249a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-cc398497-46a6-4962-86b7-a67b46f2249a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:31.759067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:31.760075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:31.760933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 436/884] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:30:31 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:31.761751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab954c73-8c6b-409e-abc8-0f6d4044de8f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:31.761751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab954c73-8c6b-409e-abc8-0f6d4044de8f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:31.761751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab954c73-8c6b-409e-abc8-0f6d4044de8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:31.779287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ab954c73-8c6b-409e-abc8-0f6d4044de8f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:31.780460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ab954c73-8c6b-409e-abc8-0f6d4044de8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:31.833769 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:31.844787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e69f2d-0f1a-4deb-a87d-804c28c5f0eb None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:31.847827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 437/885] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:31 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:31.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab954c73-8c6b-409e-abc8-0f6d4044de8f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:31.888453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a38555-54bf-4b30-8568-aec7d952751b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:31.889092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a38555-54bf-4b30-8568-aec7d952751b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:31.889367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a38555-54bf-4b30-8568-aec7d952751b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:31.889640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a38555-54bf-4b30-8568-aec7d952751b None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:31.890263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a38555-54bf-4b30-8568-aec7d952751b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:31.890564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a38555-54bf-4b30-8568-aec7d952751b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:31.892891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a38555-54bf-4b30-8568-aec7d952751b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:31.898812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 449/886] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Sep 6 22:30:31 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:31.900333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57671d0-93df-4b2e-8988-cf1ac63912f2 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:31.900616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57671d0-93df-4b2e-8988-cf1ac63912f2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:31.900934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57671d0-93df-4b2e-8988-cf1ac63912f2 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:31.904882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 450/887] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:31.905846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57671d0-93df-4b2e-8988-cf1ac63912f2 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:31.939289 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:31.968804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab954c73-8c6b-409e-abc8-0f6d4044de8f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:32.006281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890d5114-83bd-4cfb-875c-c45edf4d752e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:32.007015 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d5114-83bd-4cfb-875c-c45edf4d752e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:32.007125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d5114-83bd-4cfb-875c-c45edf4d752e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:32.007506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890d5114-83bd-4cfb-875c-c45edf4d752e None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:32.008253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d5114-83bd-4cfb-875c-c45edf4d752e None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:32.008551 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d5114-83bd-4cfb-875c-c45edf4d752e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:32.009927 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890d5114-83bd-4cfb-875c-c45edf4d752e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:32.020413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 438/888] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:30:31 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:32.020413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67982bb7-9f28-4004-8480-1f76497b3efe None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:32.020413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67982bb7-9f28-4004-8480-1f76497b3efe None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:32.020413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67982bb7-9f28-4004-8480-1f76497b3efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:32.022596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57671d0-93df-4b2e-8988-cf1ac63912f2 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:32.035960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-67982bb7-9f28-4004-8480-1f76497b3efe None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:32.037160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-67982bb7-9f28-4004-8480-1f76497b3efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:32.058435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfdcc3-adf6-4520-a7bb-26141acf6cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:32.059085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfdcc3-adf6-4520-a7bb-26141acf6cee None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:32.059362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfdcc3-adf6-4520-a7bb-26141acf6cee None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:32.059624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35dfdcc3-adf6-4520-a7bb-26141acf6cee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:32.060301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfdcc3-adf6-4520-a7bb-26141acf6cee None admin] RBAC: Adding query filter params (interface=public, service_id=8cb777efd367424ab0e0b9f4ee44db42, region_id=RegionOne) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:32.060598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfdcc3-adf6-4520-a7bb-26141acf6cee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:32.061968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35dfdcc3-adf6-4520-a7bb-26141acf6cee None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:32.067608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 451/889] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Sep 6 22:30:31 2022] GET /identity/v3/endpoints?service_id=8cb777efd367424ab0e0b9f4ee44db42&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:32.068483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfdcc3-adf6-4520-a7bb-26141acf6cee None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:32.092689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:32.116627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 439/890] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:32 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:32.117813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67982bb7-9f28-4004-8480-1f76497b3efe None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:32.178061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:32.184779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35dfdcc3-adf6-4520-a7bb-26141acf6cee None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:32.226010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c3a24-b5c6-40b3-8ea6-827cba09b2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:32.226731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3a24-b5c6-40b3-8ea6-827cba09b2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:32.227000 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3a24-b5c6-40b3-8ea6-827cba09b2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:32.227267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6c3a24-b5c6-40b3-8ea6-827cba09b2b1 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:32.227884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c3a24-b5c6-40b3-8ea6-827cba09b2b1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:32.228179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c3a24-b5c6-40b3-8ea6-827cba09b2b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:32.229790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6c3a24-b5c6-40b3-8ea6-827cba09b2b1 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:32.232739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67982bb7-9f28-4004-8480-1f76497b3efe None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:32.239208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 452/891] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:30:32 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:32.239208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c3a24-b5c6-40b3-8ea6-827cba09b2b1 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:32.275466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e1a92-ec64-4cac-b23e-bba191c9a537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:32.276070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e1a92-ec64-4cac-b23e-bba191c9a537 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:32.276343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e1a92-ec64-4cac-b23e-bba191c9a537 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:32.276607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85e1a92-ec64-4cac-b23e-bba191c9a537 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:32.277231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e1a92-ec64-4cac-b23e-bba191c9a537 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:32.278586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85e1a92-ec64-4cac-b23e-bba191c9a537 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:32.292785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 440/892] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Sep 6 22:30:32 2022] GET /identity/v3/services => generated 1268 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:30:32.292785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e1a92-ec64-4cac-b23e-bba191c9a537 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:32.352004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:32.385721 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6c3a24-b5c6-40b3-8ea6-827cba09b2b1 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:32.423106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85e1a92-ec64-4cac-b23e-bba191c9a537 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:32.428636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a5c79-1fb3-4008-b87c-9882f52698d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:32.429341 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5c79-1fb3-4008-b87c-9882f52698d9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:32.429618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5c79-1fb3-4008-b87c-9882f52698d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:32.429888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00a5c79-1fb3-4008-b87c-9882f52698d9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:32.430482 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a5c79-1fb3-4008-b87c-9882f52698d9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:32.431944 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00a5c79-1fb3-4008-b87c-9882f52698d9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-f00a5c79-1fb3-4008-b87c-9882f52698d9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:32.442114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:32.443194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:32.444026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 453/893] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Sep 6 22:30:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:32.444868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a5c79-1fb3-4008-b87c-9882f52698d9 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:32.470791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c765d1-b745-483f-91e7-3295da7f4bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:32.471423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c765d1-b745-483f-91e7-3295da7f4bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:32.471695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c765d1-b745-483f-91e7-3295da7f4bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:32.472815 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c765d1-b745-483f-91e7-3295da7f4bc7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:32.473827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:32.473941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c765d1-b745-483f-91e7-3295da7f4bc7 None admin] RBAC: Adding query filter params (interface=public, service_id=e7e7a5c447424559a214185dba366595, region_id=RegionOne) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:32.474019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c765d1-b745-483f-91e7-3295da7f4bc7 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:32.475464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c765d1-b745-483f-91e7-3295da7f4bc7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:32.483190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 441/894] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Sep 6 22:30:32 2022] GET /identity/v3/endpoints?service_id=e7e7a5c447424559a214185dba366595&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:32.484486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c765d1-b745-483f-91e7-3295da7f4bc7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:32.562466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00a5c79-1fb3-4008-b87c-9882f52698d9 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:32.599761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c765d1-b745-483f-91e7-3295da7f4bc7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:32.601182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7d0d0-582a-4967-9354-3244055bf1ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:32.601807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7d0d0-582a-4967-9354-3244055bf1ce None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:32.602135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7d0d0-582a-4967-9354-3244055bf1ce None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:32.602355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7d0d0-582a-4967-9354-3244055bf1ce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:32.611223 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:32.614070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7d0d0-582a-4967-9354-3244055bf1ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-80e7d0d0-582a-4967-9354-3244055bf1ce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:32.620679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:32.621673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:32.622527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 454/895] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Sep 6 22:30:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:32.623232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7d0d0-582a-4967-9354-3244055bf1ce None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:32.650501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441db48-c7d6-4a81-814c-e1b4a271b05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:32.651113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441db48-c7d6-4a81-814c-e1b4a271b05d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:32.651404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441db48-c7d6-4a81-814c-e1b4a271b05d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:32.651657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4441db48-c7d6-4a81-814c-e1b4a271b05d None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:32.652281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441db48-c7d6-4a81-814c-e1b4a271b05d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:32.652575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441db48-c7d6-4a81-814c-e1b4a271b05d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:32.653964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4441db48-c7d6-4a81-814c-e1b4a271b05d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:32.658641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 442/896] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Sep 6 22:30:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:32.659545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441db48-c7d6-4a81-814c-e1b4a271b05d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:32.718361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:32.737344 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7d0d0-582a-4967-9354-3244055bf1ce None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:32.776251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d030c5-ba1e-49c4-a01a-1042f5bf14dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:32.776997 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d030c5-ba1e-49c4-a01a-1042f5bf14dd None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:32.777267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d030c5-ba1e-49c4-a01a-1042f5bf14dd None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:32.777531 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d030c5-ba1e-49c4-a01a-1042f5bf14dd None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:32.778131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d030c5-ba1e-49c4-a01a-1042f5bf14dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:32.779475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d030c5-ba1e-49c4-a01a-1042f5bf14dd None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:32.780143 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4441db48-c7d6-4a81-814c-e1b4a271b05d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:32.786060 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 455/897] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Sep 6 22:30:32 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:30:32.786927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d030c5-ba1e-49c4-a01a-1042f5bf14dd None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:32.856420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573114ba-a527-4499-83c0-9cb40589ba65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:32.857372 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573114ba-a527-4499-83c0-9cb40589ba65 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:32.857979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573114ba-a527-4499-83c0-9cb40589ba65 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:32.858393 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573114ba-a527-4499-83c0-9cb40589ba65 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:32.859218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573114ba-a527-4499-83c0-9cb40589ba65 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:32.860171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573114ba-a527-4499-83c0-9cb40589ba65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:32.862414 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573114ba-a527-4499-83c0-9cb40589ba65 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:32.877823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 443/898] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Sep 6 22:30:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:32.884558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573114ba-a527-4499-83c0-9cb40589ba65 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:32.968236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:33.032212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d030c5-ba1e-49c4-a01a-1042f5bf14dd None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:33.088784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573114ba-a527-4499-83c0-9cb40589ba65 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:33.089410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8717f-85b2-438e-9bac-aa37d412823d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:33.090010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8717f-85b2-438e-9bac-aa37d412823d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:33.090461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8717f-85b2-438e-9bac-aa37d412823d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:33.090814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b8717f-85b2-438e-9bac-aa37d412823d None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:33.096786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8717f-85b2-438e-9bac-aa37d412823d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:33.096786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8717f-85b2-438e-9bac-aa37d412823d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:33.096786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b8717f-85b2-438e-9bac-aa37d412823d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:33.100712 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 456/899] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Sep 6 22:30:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:33.158327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52294c4-39a7-41a4-814f-0bca6b9047b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:33.158946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52294c4-39a7-41a4-814f-0bca6b9047b2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:33.159223 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52294c4-39a7-41a4-814f-0bca6b9047b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:33.159533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52294c4-39a7-41a4-814f-0bca6b9047b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:33.172503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52294c4-39a7-41a4-814f-0bca6b9047b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:33.174049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52294c4-39a7-41a4-814f-0bca6b9047b2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-d52294c4-39a7-41a4-814f-0bca6b9047b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:33.195439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:33.196438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:33.197369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:33.197369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:33.197369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:33.197369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 444/900] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:33.204227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8717f-85b2-438e-9bac-aa37d412823d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:33.261264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:33.366139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b8717f-85b2-438e-9bac-aa37d412823d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:33.424249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373a681-4eee-4f0f-945f-94df72ef442b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:33.426361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373a681-4eee-4f0f-945f-94df72ef442b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:33.426867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373a681-4eee-4f0f-945f-94df72ef442b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:33.427338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373a681-4eee-4f0f-945f-94df72ef442b None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:33.428177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373a681-4eee-4f0f-945f-94df72ef442b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:33.428674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373a681-4eee-4f0f-945f-94df72ef442b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:33.436784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373a681-4eee-4f0f-945f-94df72ef442b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:33.456959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 457/901] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:33.464512 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52294c4-39a7-41a4-814f-0bca6b9047b2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:33.590800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:33.606078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52294c4-39a7-41a4-814f-0bca6b9047b2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:33.648234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c55ee-fd84-4a7e-93ff-6049c1ba4230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:33.648913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c55ee-fd84-4a7e-93ff-6049c1ba4230 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:33.649196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c55ee-fd84-4a7e-93ff-6049c1ba4230 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:33.649474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7c55ee-fd84-4a7e-93ff-6049c1ba4230 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:33.668827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c55ee-fd84-4a7e-93ff-6049c1ba4230 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:33.669969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7c55ee-fd84-4a7e-93ff-6049c1ba4230 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-0f7c55ee-fd84-4a7e-93ff-6049c1ba4230 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:33.686572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:33.687524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:33.688459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:33.688459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:33.688459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:33.688459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:33.688459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:33.688459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:33.688459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:33.688459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:33.688459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:33.688459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:33.692188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 445/902] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Sep 6 22:30:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:33.701871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373a681-4eee-4f0f-945f-94df72ef442b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:33.844334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:33.881356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373a681-4eee-4f0f-945f-94df72ef442b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:33.921031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13a2b8-367c-4a8d-ab8d-b7eda719edb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:33.921806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13a2b8-367c-4a8d-ab8d-b7eda719edb9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:33.922185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13a2b8-367c-4a8d-ab8d-b7eda719edb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:33.922556 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13a2b8-367c-4a8d-ab8d-b7eda719edb9 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:33.924904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13a2b8-367c-4a8d-ab8d-b7eda719edb9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:33.924904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13a2b8-367c-4a8d-ab8d-b7eda719edb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:33.926302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13a2b8-367c-4a8d-ab8d-b7eda719edb9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:33.947329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 458/903] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Sep 6 22:30:33 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:33.953083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c55ee-fd84-4a7e-93ff-6049c1ba4230 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:34.066449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7c55ee-fd84-4a7e-93ff-6049c1ba4230 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:34.101574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e028738-549c-40df-b05f-4ac700afa46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:34.103400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e028738-549c-40df-b05f-4ac700afa46e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:34.103400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e028738-549c-40df-b05f-4ac700afa46e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:34.103400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e028738-549c-40df-b05f-4ac700afa46e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:34.152914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:34.194914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e028738-549c-40df-b05f-4ac700afa46e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:34.196379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e028738-549c-40df-b05f-4ac700afa46e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:34.303594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 446/904] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:33 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:30:34.311338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13a2b8-367c-4a8d-ab8d-b7eda719edb9 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:34.416822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13a2b8-367c-4a8d-ab8d-b7eda719edb9 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:34.455967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:34.458170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c56943-907e-4824-89e0-f9416fed6974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:34.459117 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c56943-907e-4824-89e0-f9416fed6974 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:34.459401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c56943-907e-4824-89e0-f9416fed6974 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:34.459674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c56943-907e-4824-89e0-f9416fed6974 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:34.471114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c56943-907e-4824-89e0-f9416fed6974 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:34.472601 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c56943-907e-4824-89e0-f9416fed6974 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-e4c56943-907e-4824-89e0-f9416fed6974 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:34.489205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:34.490176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:34.491083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:34.491083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:34.491083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:34.491083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 459/905] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:34.497245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e028738-549c-40df-b05f-4ac700afa46e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:34.557914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:34.614685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e028738-549c-40df-b05f-4ac700afa46e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:34.651392 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942d6ff-c8cd-4fb4-91a1-5976c5dd488e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:34.652070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942d6ff-c8cd-4fb4-91a1-5976c5dd488e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:34.652352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942d6ff-c8cd-4fb4-91a1-5976c5dd488e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:34.652622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e942d6ff-c8cd-4fb4-91a1-5976c5dd488e None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:34.653312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942d6ff-c8cd-4fb4-91a1-5976c5dd488e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:34.653795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942d6ff-c8cd-4fb4-91a1-5976c5dd488e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:34.655327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e942d6ff-c8cd-4fb4-91a1-5976c5dd488e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:34.667615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613c09d-6429-488c-ab2d-6bc03d4fa6e8 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:34.668212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613c09d-6429-488c-ab2d-6bc03d4fa6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:34.668678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613c09d-6429-488c-ab2d-6bc03d4fa6e8 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:34.669954 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 447/906] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:34.675578 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942d6ff-c8cd-4fb4-91a1-5976c5dd488e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:34.675845 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 460/907] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:30:34.677520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df111244-13c0-4346-92bc-093df34291c2 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:34.678021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df111244-13c0-4346-92bc-093df34291c2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:34.678500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df111244-13c0-4346-92bc-093df34291c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:34.714409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-df111244-13c0-4346-92bc-093df34291c2 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:34.715945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-df111244-13c0-4346-92bc-093df34291c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:34.799517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e942d6ff-c8cd-4fb4-91a1-5976c5dd488e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:34.813201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:34.847052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a3109-f844-42af-95ca-eac5da929b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:34.848100 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3109-f844-42af-95ca-eac5da929b8d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:34.848714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3109-f844-42af-95ca-eac5da929b8d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:34.849161 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624a3109-f844-42af-95ca-eac5da929b8d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:34.863735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a3109-f844-42af-95ca-eac5da929b8d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:34.865709 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624a3109-f844-42af-95ca-eac5da929b8d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-624a3109-f844-42af-95ca-eac5da929b8d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:34.883567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:34.884687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:34.885890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:34.885890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:34.885890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:34.885890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:34.885890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:34.885890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:34.885890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:34.885890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:34.892669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 448/909] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:34.894371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a3109-f844-42af-95ca-eac5da929b8d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:34.896190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 461/909] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:34 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:35.005783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624a3109-f844-42af-95ca-eac5da929b8d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:35.045721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4129ca5-5b4a-4152-847a-9a9b267f68a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:35.046685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4129ca5-5b4a-4152-847a-9a9b267f68a3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:35.047195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4129ca5-5b4a-4152-847a-9a9b267f68a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:35.047682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4129ca5-5b4a-4152-847a-9a9b267f68a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:35.048596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4129ca5-5b4a-4152-847a-9a9b267f68a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:35.049140 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4129ca5-5b4a-4152-847a-9a9b267f68a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:35.050976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4129ca5-5b4a-4152-847a-9a9b267f68a3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:35.064431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 449/910] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:35.074038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df111244-13c0-4346-92bc-093df34291c2 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:35.185264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df111244-13c0-4346-92bc-093df34291c2 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:35.204150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:35.224986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8856735-c941-44aa-90cd-6fdd7db2accf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:35.226079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8856735-c941-44aa-90cd-6fdd7db2accf None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:35.226561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8856735-c941-44aa-90cd-6fdd7db2accf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:35.227034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8856735-c941-44aa-90cd-6fdd7db2accf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:35.227994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8856735-c941-44aa-90cd-6fdd7db2accf None admin] RBAC: Adding query filter params (role.id=a2aaf3bf584d4607946a010016852567, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=7da74f16cb9b4b15a902947103a25cf6) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:35.228542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8856735-c941-44aa-90cd-6fdd7db2accf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:35.230259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8856735-c941-44aa-90cd-6fdd7db2accf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:35.238656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 462/911] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:35 2022] GET /identity/v3/role_assignments?user.id=7da74f16cb9b4b15a902947103a25cf6&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=a2aaf3bf584d4607946a010016852567 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:35.272483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4129ca5-5b4a-4152-847a-9a9b267f68a3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:35.329317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:35.386515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5563-6636-40d1-8aec-4ce22181d3ca None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:35.387041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5563-6636-40d1-8aec-4ce22181d3ca None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:35.387041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcf5563-6636-40d1-8aec-4ce22181d3ca None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:35.388709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 463/912] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:35.395356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fd201-2afc-4083-b60a-65e0c08f9213 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:35.396053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fd201-2afc-4083-b60a-65e0c08f9213 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:35.396546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869fd201-2afc-4083-b60a-65e0c08f9213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:35.423251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4129ca5-5b4a-4152-847a-9a9b267f68a3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:35.429185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-869fd201-2afc-4083-b60a-65e0c08f9213 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:35.430689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-869fd201-2afc-4083-b60a-65e0c08f9213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:35.471233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa20e73-c521-4225-8067-6501e02adf43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:35.472215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa20e73-c521-4225-8067-6501e02adf43 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:35.472635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa20e73-c521-4225-8067-6501e02adf43 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:35.473110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa20e73-c521-4225-8067-6501e02adf43 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:35.474042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa20e73-c521-4225-8067-6501e02adf43 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:35.475597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa20e73-c521-4225-8067-6501e02adf43 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-dfa20e73-c521-4225-8067-6501e02adf43 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:35.487019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:35.488032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:35.488970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 450/913] 10.222.0.39 () {56 vars in 1132 bytes} [Tue Sep 6 22:30:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:35.499202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa20e73-c521-4225-8067-6501e02adf43 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:35.549005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 464/914] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:35.647543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:35.657543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa20e73-c521-4225-8067-6501e02adf43 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:35.677123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57c195-70c6-49cd-8cdb-f6ea33d98a19 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:35.677719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57c195-70c6-49cd-8cdb-f6ea33d98a19 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:35.678208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e57c195-70c6-49cd-8cdb-f6ea33d98a19 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:35.681041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 465/915] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:35.684653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c107ba-a867-43c2-aa33-8acc3834b871 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:35.685229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c107ba-a867-43c2-aa33-8acc3834b871 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:35.685695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c107ba-a867-43c2-aa33-8acc3834b871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:35.698118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbc841-9c5e-4de5-9b3d-0f45e345677f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:35.698730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbc841-9c5e-4de5-9b3d-0f45e345677f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:35.699008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbc841-9c5e-4de5-9b3d-0f45e345677f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:35.699281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbc841-9c5e-4de5-9b3d-0f45e345677f None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:35.699904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbc841-9c5e-4de5-9b3d-0f45e345677f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:35.700203 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbc841-9c5e-4de5-9b3d-0f45e345677f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:35.701624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbc841-9c5e-4de5-9b3d-0f45e345677f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:35.707862 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 451/916] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Sep 6 22:30:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:35.714621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbc841-9c5e-4de5-9b3d-0f45e345677f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:35.719673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-26c107ba-a867-43c2-aa33-8acc3834b871 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:35.722596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-26c107ba-a867-43c2-aa33-8acc3834b871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:35.842114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbc841-9c5e-4de5-9b3d-0f45e345677f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:35.857552 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 466/917] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:35 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:35.868315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c107ba-a867-43c2-aa33-8acc3834b871 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:35.885859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dbd8e-2c41-442a-9c5a-ffca628a64fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:35.886736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dbd8e-2c41-442a-9c5a-ffca628a64fc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:35.887216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dbd8e-2c41-442a-9c5a-ffca628a64fc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:35.887702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08dbd8e-2c41-442a-9c5a-ffca628a64fc None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:35.888993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dbd8e-2c41-442a-9c5a-ffca628a64fc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:35.889648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dbd8e-2c41-442a-9c5a-ffca628a64fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:35.891229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08dbd8e-2c41-442a-9c5a-ffca628a64fc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:35.902649 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 452/918] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Sep 6 22:30:35 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:35.903746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dbd8e-2c41-442a-9c5a-ffca628a64fc None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:35.923284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:35.932885 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:36.028649 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08dbd8e-2c41-442a-9c5a-ffca628a64fc None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:36.038853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c107ba-a867-43c2-aa33-8acc3834b871 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:36.081861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3e0ff-789b-4bcb-ae81-cf009334fae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:36.083243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3e0ff-789b-4bcb-ae81-cf009334fae9 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:36.084276 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3e0ff-789b-4bcb-ae81-cf009334fae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:36.085111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3e0ff-789b-4bcb-ae81-cf009334fae9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:36.086422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3e0ff-789b-4bcb-ae81-cf009334fae9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:36.089113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a3e0ff-789b-4bcb-ae81-cf009334fae9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:36.094632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b12a4-6e55-4cd0-a2d6-f35530956f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:36.095552 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b12a4-6e55-4cd0-a2d6-f35530956f0f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:36.096338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b12a4-6e55-4cd0-a2d6-f35530956f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:36.097609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856b12a4-6e55-4cd0-a2d6-f35530956f0f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:36.098768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b12a4-6e55-4cd0-a2d6-f35530956f0f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:36.101313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856b12a4-6e55-4cd0-a2d6-f35530956f0f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-856b12a4-6e55-4cd0-a2d6-f35530956f0f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:36.118038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:36.119133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 06 22:30:36.120043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:36.123013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 467/919] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:30:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:36.128662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b12a4-6e55-4cd0-a2d6-f35530956f0f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:36.137330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 453/920] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:30:35 2022] POST /identity/v3/endpoints => generated 347 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:36.139357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3e0ff-789b-4bcb-ae81-cf009334fae9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:36.278329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:36.282714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856b12a4-6e55-4cd0-a2d6-f35530956f0f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:36.285854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:36.343527 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3e0ff-789b-4bcb-ae81-cf009334fae9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:36.350716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629628e-b281-4fe1-9d2b-2bc06c22ee1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:36.351754 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629628e-b281-4fe1-9d2b-2bc06c22ee1f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:36.352248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629628e-b281-4fe1-9d2b-2bc06c22ee1f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:36.353099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629628e-b281-4fe1-9d2b-2bc06c22ee1f None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:36.353932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629628e-b281-4fe1-9d2b-2bc06c22ee1f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:36.354577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629628e-b281-4fe1-9d2b-2bc06c22ee1f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:36.356173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629628e-b281-4fe1-9d2b-2bc06c22ee1f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:36.369775 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 468/921] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:30:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:36.372598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629628e-b281-4fe1-9d2b-2bc06c22ee1f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:36.395630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328a529-9066-494f-b8be-20613525c69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:36.396243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328a529-9066-494f-b8be-20613525c69e None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:36.396517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328a529-9066-494f-b8be-20613525c69e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:36.396865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328a529-9066-494f-b8be-20613525c69e None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:36.397464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328a529-9066-494f-b8be-20613525c69e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:36.398829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328a529-9066-494f-b8be-20613525c69e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:36.421741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 454/922] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:30:36 2022] POST /identity/v3/services => generated 241 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:36.527708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629628e-b281-4fe1-9d2b-2bc06c22ee1f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:36.597988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2617edb5-6608-4512-a201-a725b44f39d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:36.598976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617edb5-6608-4512-a201-a725b44f39d6 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:36.599457 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617edb5-6608-4512-a201-a725b44f39d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:36.599964 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2617edb5-6608-4512-a201-a725b44f39d6 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:36.600844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617edb5-6608-4512-a201-a725b44f39d6 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:36.601376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617edb5-6608-4512-a201-a725b44f39d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:36.602986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2617edb5-6608-4512-a201-a725b44f39d6 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:36.610300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 469/923] 10.222.0.39 () {56 vars in 1114 bytes} [Tue Sep 6 22:30:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:36.618915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328a529-9066-494f-b8be-20613525c69e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:36.773960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328a529-9066-494f-b8be-20613525c69e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:36.813030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:36.820269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aa989f-ef6a-4f7a-8dda-1656b34e3112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:36.821116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa989f-ef6a-4f7a-8dda-1656b34e3112 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:36.821411 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa989f-ef6a-4f7a-8dda-1656b34e3112 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:36.821699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aa989f-ef6a-4f7a-8dda-1656b34e3112 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:36.822373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa989f-ef6a-4f7a-8dda-1656b34e3112 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:36.826096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aa989f-ef6a-4f7a-8dda-1656b34e3112 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:36.852540 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 455/924] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:30:36 2022] POST /identity/v3/endpoints => generated 346 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:37.747329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cd5d2-8e16-4d29-84db-da4fc04f68ed None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:37.747329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cd5d2-8e16-4d29-84db-da4fc04f68ed None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:37.747329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cd5d2-8e16-4d29-84db-da4fc04f68ed None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:37.747329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 470/925] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:37.755460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7cfd-9d02-4778-a0aa-522767af8252 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:37.755899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7cfd-9d02-4778-a0aa-522767af8252 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:37.756048 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad7cfd-9d02-4778-a0aa-522767af8252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:37.781344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-9fad7cfd-9d02-4778-a0aa-522767af8252 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:37.782869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9fad7cfd-9d02-4778-a0aa-522767af8252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:37.874721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 456/926] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:37 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:38.194171 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cd5d2-8e16-4d29-84db-da4fc04f68ed None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:38.218720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:38.304488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cd5d2-8e16-4d29-84db-da4fc04f68ed None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:38.348954 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff0c7b-a949-4e73-a3c8-0100a1fa64ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:38.349599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff0c7b-a949-4e73-a3c8-0100a1fa64ad None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:38.349873 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff0c7b-a949-4e73-a3c8-0100a1fa64ad None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:38.350140 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff0c7b-a949-4e73-a3c8-0100a1fa64ad None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:38.364484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fff0c7b-a949-4e73-a3c8-0100a1fa64ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-0fff0c7b-a949-4e73-a3c8-0100a1fa64ad None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:38.369352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:38.370312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:38.371152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 471/927] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Sep 6 22:30:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:38.379866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad7cfd-9d02-4778-a0aa-522767af8252 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:38.499149 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad7cfd-9d02-4778-a0aa-522767af8252 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:38.515797 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:38.537487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20566e73-d989-4523-a709-1179c00dcaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:38.538427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20566e73-d989-4523-a709-1179c00dcaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:38.544425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20566e73-d989-4523-a709-1179c00dcaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:38.544425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20566e73-d989-4523-a709-1179c00dcaa3 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:38.544425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20566e73-d989-4523-a709-1179c00dcaa3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:38.544425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20566e73-d989-4523-a709-1179c00dcaa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:38.544425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20566e73-d989-4523-a709-1179c00dcaa3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:38.555350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 457/928] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Sep 6 22:30:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:38.570851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff0c7b-a949-4e73-a3c8-0100a1fa64ad None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:38.688877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:38.711313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff0c7b-a949-4e73-a3c8-0100a1fa64ad None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:38.717669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d0fe5-6ec0-4dc8-bd5d-e1cb45eeb04e None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:38.718263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d0fe5-6ec0-4dc8-bd5d-e1cb45eeb04e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:38.718757 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550d0fe5-6ec0-4dc8-bd5d-e1cb45eeb04e None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:38.720571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 458/929] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:38.727259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f456f-03b3-4548-83a5-fad8d5342d26 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:38.727827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f456f-03b3-4548-83a5-fad8d5342d26 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:38.728331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f456f-03b3-4548-83a5-fad8d5342d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:38.751265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-457f456f-03b3-4548-83a5-fad8d5342d26 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:38.752792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-457f456f-03b3-4548-83a5-fad8d5342d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:38.757172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5686ec1-01aa-43e5-90db-7cf669756634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:38.757817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5686ec1-01aa-43e5-90db-7cf669756634 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:38.758339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5686ec1-01aa-43e5-90db-7cf669756634 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:38.758602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5686ec1-01aa-43e5-90db-7cf669756634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:38.777241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5686ec1-01aa-43e5-90db-7cf669756634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:38.779126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5686ec1-01aa-43e5-90db-7cf669756634 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-d5686ec1-01aa-43e5-90db-7cf669756634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:38.799603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:38.801211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:38.802111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:38.802111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:38.802111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:38.805063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 472/930] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:38.811192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5686ec1-01aa-43e5-90db-7cf669756634 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:38.916315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 459/931] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:38.916315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929bca8-56e7-4b07-8e20-1b3c4e609344 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:38.916315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929bca8-56e7-4b07-8e20-1b3c4e609344 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:38.916315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d929bca8-56e7-4b07-8e20-1b3c4e609344 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:38.922215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 460/932] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:38.923732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca4704-4a12-4c07-bad0-bbac232e53d3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:38.923979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca4704-4a12-4c07-bad0-bbac232e53d3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:38.924249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca4704-4a12-4c07-bad0-bbac232e53d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:38.948419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-17ca4704-4a12-4c07-bad0-bbac232e53d3 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:38.951697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-17ca4704-4a12-4c07-bad0-bbac232e53d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:39.027790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5686ec1-01aa-43e5-90db-7cf669756634 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:39.052817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 461/933] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:38 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:39.079026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:39.085725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653e752-489a-442f-813c-b71e4bf3c58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:39.086333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653e752-489a-442f-813c-b71e4bf3c58c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:39.086607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653e752-489a-442f-813c-b71e4bf3c58c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:39.086874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e653e752-489a-442f-813c-b71e4bf3c58c None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:39.087579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653e752-489a-442f-813c-b71e4bf3c58c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:39.087878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653e752-489a-442f-813c-b71e4bf3c58c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:39.089529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e653e752-489a-442f-813c-b71e4bf3c58c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:39.103439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 473/934] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:39.109381 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca4704-4a12-4c07-bad0-bbac232e53d3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:39.174013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:39.215886 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653e752-489a-442f-813c-b71e4bf3c58c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:39.265653 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca4704-4a12-4c07-bad0-bbac232e53d3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:39.307322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969e04b-046f-498f-942e-79bcb0fc4486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:39.308184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969e04b-046f-498f-942e-79bcb0fc4486 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:39.308625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969e04b-046f-498f-942e-79bcb0fc4486 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:39.309063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3969e04b-046f-498f-942e-79bcb0fc4486 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:39.317032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969e04b-046f-498f-942e-79bcb0fc4486 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:39.318828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3969e04b-046f-498f-942e-79bcb0fc4486 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-3969e04b-046f-498f-942e-79bcb0fc4486 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:39.333222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:39.334193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:39.335228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:39.335228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:39.335228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:39.335228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:39.335228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:39.335228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:39.335228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:39.335228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:39.335228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:39.335228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:39.337728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e653e752-489a-442f-813c-b71e4bf3c58c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:39.337957 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 462/935] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Sep 6 22:30:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:39.343910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969e04b-046f-498f-942e-79bcb0fc4486 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:39.380411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5425f1-3379-416c-88c3-1614736eb7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:39.381087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5425f1-3379-416c-88c3-1614736eb7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:39.384062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5425f1-3379-416c-88c3-1614736eb7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:39.384062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5425f1-3379-416c-88c3-1614736eb7d1 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:39.384062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5425f1-3379-416c-88c3-1614736eb7d1 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:39.384898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5425f1-3379-416c-88c3-1614736eb7d1 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-cc5425f1-3379-416c-88c3-1614736eb7d1 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:39.396650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:39.397932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 22:30:39.398793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:39.402077 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 474/936] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:39 2022] GET /identity/v3/services/compute => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:39.403270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf80d7-4aad-4311-a126-859e5c0328e4 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:39.403559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf80d7-4aad-4311-a126-859e5c0328e4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:39.403836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf80d7-4aad-4311-a126-859e5c0328e4 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:39.406385 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 475/937] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:39.406385 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf80d7-4aad-4311-a126-859e5c0328e4 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:39.463182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:39.469831 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3969e04b-046f-498f-942e-79bcb0fc4486 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:39.473312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:39.507326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e55ac4-c154-4ad9-8465-7e83d542b019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:39.507933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e55ac4-c154-4ad9-8465-7e83d542b019 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:39.508458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e55ac4-c154-4ad9-8465-7e83d542b019 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:39.508724 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e55ac4-c154-4ad9-8465-7e83d542b019 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:39.509513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e55ac4-c154-4ad9-8465-7e83d542b019 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:39.509783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e55ac4-c154-4ad9-8465-7e83d542b019 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:39.511209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e55ac4-c154-4ad9-8465-7e83d542b019 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:39.524056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf80d7-4aad-4311-a126-859e5c0328e4 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:39.533231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 463/938] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Sep 6 22:30:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:39.533231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e55ac4-c154-4ad9-8465-7e83d542b019 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:39.568945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8e9c6e-28bb-4142-83f2-c455d2c77fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:39.569508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e9c6e-28bb-4142-83f2-c455d2c77fcf None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:39.569778 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e9c6e-28bb-4142-83f2-c455d2c77fcf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:39.570055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8e9c6e-28bb-4142-83f2-c455d2c77fcf None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:39.572818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e9c6e-28bb-4142-83f2-c455d2c77fcf None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:39.572818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8e9c6e-28bb-4142-83f2-c455d2c77fcf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-5e8e9c6e-28bb-4142-83f2-c455d2c77fcf None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:39.583235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:39.584239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:39.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 476/939] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Sep 6 22:30:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:39.590702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24d179-af0d-45c4-a714-c52d60f23f35 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:39.590702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24d179-af0d-45c4-a714-c52d60f23f35 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:39.590702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c24d179-af0d-45c4-a714-c52d60f23f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:39.607612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-1c24d179-af0d-45c4-a714-c52d60f23f35 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:39.608825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1c24d179-af0d-45c4-a714-c52d60f23f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:39.645098 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e55ac4-c154-4ad9-8465-7e83d542b019 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:39.682980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c3247-de40-401f-ac60-cf7b0a00ed24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:39.683592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c3247-de40-401f-ac60-cf7b0a00ed24 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:39.683867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c3247-de40-401f-ac60-cf7b0a00ed24 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:39.684164 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c3247-de40-401f-ac60-cf7b0a00ed24 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:39.684849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c3247-de40-401f-ac60-cf7b0a00ed24 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:39.685157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c3247-de40-401f-ac60-cf7b0a00ed24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:39.686513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c3247-de40-401f-ac60-cf7b0a00ed24 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:39.689182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 477/940] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:39 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:39.690310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24d179-af0d-45c4-a714-c52d60f23f35 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:39.693008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 464/941] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:39 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:39.693886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c3247-de40-401f-ac60-cf7b0a00ed24 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:39.721094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:39.765438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:39.830728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c3247-de40-401f-ac60-cf7b0a00ed24 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:39.836562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c24d179-af0d-45c4-a714-c52d60f23f35 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:39.875926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad97e0-06be-4d89-9f73-3e47f987f9df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:39.876577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad97e0-06be-4d89-9f73-3e47f987f9df None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:39.876865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad97e0-06be-4d89-9f73-3e47f987f9df None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:39.877143 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ad97e0-06be-4d89-9f73-3e47f987f9df None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:39.877772 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad97e0-06be-4d89-9f73-3e47f987f9df None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:39.878635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad97e0-06be-4d89-9f73-3e47f987f9df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:39.880061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ad97e0-06be-4d89-9f73-3e47f987f9df None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:39.883435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6b659-31b1-4d3e-9f0e-31993fd21139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:39.884354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6b659-31b1-4d3e-9f0e-31993fd21139 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:39.884737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6b659-31b1-4d3e-9f0e-31993fd21139 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:39.884935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb6b659-31b1-4d3e-9f0e-31993fd21139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:39.886717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 465/942] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Sep 6 22:30:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:39.887702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad97e0-06be-4d89-9f73-3e47f987f9df None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:39.901022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6b659-31b1-4d3e-9f0e-31993fd21139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:39.903292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb6b659-31b1-4d3e-9f0e-31993fd21139 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-fcb6b659-31b1-4d3e-9f0e-31993fd21139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:39.926453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:39.927811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:39.929294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:39.929294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:39.929294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:39.929294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 478/943] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:39.929294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6b659-31b1-4d3e-9f0e-31993fd21139 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:39.998186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:40.018399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ad97e0-06be-4d89-9f73-3e47f987f9df None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:40.056141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb6b659-31b1-4d3e-9f0e-31993fd21139 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:40.057761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:40.059870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eee5f0-2686-4733-ac89-c7e808a566d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:40.060477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eee5f0-2686-4733-ac89-c7e808a566d1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:40.060788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eee5f0-2686-4733-ac89-c7e808a566d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:40.061058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eee5f0-2686-4733-ac89-c7e808a566d1 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:40.061689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eee5f0-2686-4733-ac89-c7e808a566d1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:40.061987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eee5f0-2686-4733-ac89-c7e808a566d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:40.063348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eee5f0-2686-4733-ac89-c7e808a566d1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:40.070813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 466/944] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:39 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:40.071683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eee5f0-2686-4733-ac89-c7e808a566d1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:40.095350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77339663-9886-42f5-9ae0-1993a645e4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:40.095988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339663-9886-42f5-9ae0-1993a645e4db None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:40.096254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339663-9886-42f5-9ae0-1993a645e4db None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:40.096542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77339663-9886-42f5-9ae0-1993a645e4db None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:40.097263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77339663-9886-42f5-9ae0-1993a645e4db None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:40.097570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77339663-9886-42f5-9ae0-1993a645e4db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:40.099165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77339663-9886-42f5-9ae0-1993a645e4db None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:40.110338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 479/945] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Sep 6 22:30:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:40.111350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77339663-9886-42f5-9ae0-1993a645e4db None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:40.220682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eee5f0-2686-4733-ac89-c7e808a566d1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:40.256341 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77339663-9886-42f5-9ae0-1993a645e4db None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:40.272082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ce948-556d-48b6-a464-f902aa737c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:40.273022 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ce948-556d-48b6-a464-f902aa737c42 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:40.273294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ce948-556d-48b6-a464-f902aa737c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:40.273838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864ce948-556d-48b6-a464-f902aa737c42 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:40.274208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ce948-556d-48b6-a464-f902aa737c42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:40.274577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ce948-556d-48b6-a464-f902aa737c42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:40.275884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864ce948-556d-48b6-a464-f902aa737c42 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:40.296917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 467/946] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:40.298087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ce948-556d-48b6-a464-f902aa737c42 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:40.308753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:40.310338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73be332-96b5-4a76-aa5a-7a49e370e979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:40.311020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73be332-96b5-4a76-aa5a-7a49e370e979 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:40.311301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73be332-96b5-4a76-aa5a-7a49e370e979 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:40.311575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73be332-96b5-4a76-aa5a-7a49e370e979 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:40.312226 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73be332-96b5-4a76-aa5a-7a49e370e979 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:40.313718 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73be332-96b5-4a76-aa5a-7a49e370e979 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:40.317017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: WARNING keystone.server.flask.common [None req-b73be332-96b5-4a76-aa5a-7a49e370e979 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 22:30:40.341021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.sql.core [None req-b73be332-96b5-4a76-aa5a-7a49e370e979 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 22:30:40.341021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 22:30:40.341021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [parameters: {'id': 'c5643937cd1d4f5480999b0fdfada061', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 22:30:40.341021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81666) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-b73be332-96b5-4a76-aa5a-7a49e370e979 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c5643937cd1d4f5480999b0fdfada061.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c5643937cd1d4f5480999b0fdfada061. Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application conn.query(q) Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 22:30:40.364189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application result.read() Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 06 22:30:40.365158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application next(self.gen) Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application yield resource Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application next(self.gen) Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise self.value Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application session.commit() Sep 06 22:30:40.371580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self.session.flush() Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application compat.raise_( Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 22:30:40.372503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 22:30:40.373565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application conn.query(q) Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application result.read() Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 22:30:40.374372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [parameters: {'id': 'c5643937cd1d4f5480999b0fdfada061', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:40.375281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:40.378965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:40.381173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:40.381173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 06 22:30:40.381173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 06 22:30:40.381173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:40.381173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:40.381173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 06 22:30:40.381173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 06 22:30:40.381173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id c5643937cd1d4f5480999b0fdfada061. Sep 06 22:30:40.381173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:40.381173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 480/947] 10.222.0.39 () {60 vars in 1140 bytes} [Tue Sep 6 22:30:40 2022] POST /identity/v3/projects => generated 262 bytes in 255 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:40.381173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73be332-96b5-4a76-aa5a-7a49e370e979 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:40.406316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:40.434882 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864ce948-556d-48b6-a464-f902aa737c42 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:40.484209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2f625-00c6-4aa6-81cf-20e9a26ce218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:40.485418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2f625-00c6-4aa6-81cf-20e9a26ce218 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:40.485852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2f625-00c6-4aa6-81cf-20e9a26ce218 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:40.486273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada2f625-00c6-4aa6-81cf-20e9a26ce218 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:40.487914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2f625-00c6-4aa6-81cf-20e9a26ce218 None admin] RBAC: Adding query filter params (interface=public, service_id=d2ae420d07934b5eb08a96c65b69c8e6, region_id=RegionOne) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:40.488425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2f625-00c6-4aa6-81cf-20e9a26ce218 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:40.490493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada2f625-00c6-4aa6-81cf-20e9a26ce218 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:40.496667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 468/948] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Sep 6 22:30:40 2022] GET /identity/v3/endpoints?service_id=d2ae420d07934b5eb08a96c65b69c8e6&interface=public®ion_id=RegionOne => generated 194 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:40.498003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2f625-00c6-4aa6-81cf-20e9a26ce218 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:40.515268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73be332-96b5-4a76-aa5a-7a49e370e979 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:40.562165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03940565-0236-4dd0-b706-82f2437a9f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:40.563103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03940565-0236-4dd0-b706-82f2437a9f71 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:40.563717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03940565-0236-4dd0-b706-82f2437a9f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:40.564189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03940565-0236-4dd0-b706-82f2437a9f71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:40.575336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03940565-0236-4dd0-b706-82f2437a9f71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:40.575336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03940565-0236-4dd0-b706-82f2437a9f71 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-03940565-0236-4dd0-b706-82f2437a9f71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:40.590646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:40.591591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:40.592514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:40.592514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:40.592514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:40.592514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:40.592514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:40.592514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:40.592514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:40.592514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:40.596681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 481/949] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:40.598359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03940565-0236-4dd0-b706-82f2437a9f71 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:40.603824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:40.655769 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada2f625-00c6-4aa6-81cf-20e9a26ce218 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:40.701488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ffe13-9be1-4ce1-bcb4-e5cef69b27f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:40.701866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:40.702690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ffe13-9be1-4ce1-bcb4-e5cef69b27f7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:40.703162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ffe13-9be1-4ce1-bcb4-e5cef69b27f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:40.703626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975ffe13-9be1-4ce1-bcb4-e5cef69b27f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:40.713783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ffe13-9be1-4ce1-bcb4-e5cef69b27f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:40.715929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975ffe13-9be1-4ce1-bcb4-e5cef69b27f7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:40.725749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03940565-0236-4dd0-b706-82f2437a9f71 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-975ffe13-9be1-4ce1-bcb4-e5cef69b27f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:40.731916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:40.732839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:40.733712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:40.733712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:40.733712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:40.733712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:40.733712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:40.733712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:40.733712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:40.733712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:40.738261 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 469/950] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:40.739339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ffe13-9be1-4ce1-bcb4-e5cef69b27f7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:40.782016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa41c42-2981-472a-a91b-de4e1075111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:40.782658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41c42-2981-472a-a91b-de4e1075111e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:40.782879 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41c42-2981-472a-a91b-de4e1075111e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:40.783152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa41c42-2981-472a-a91b-de4e1075111e None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:40.783752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa41c42-2981-472a-a91b-de4e1075111e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:40.786442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa41c42-2981-472a-a91b-de4e1075111e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:40.805115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 482/951] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Sep 6 22:30:40 2022] GET /identity/v3/services => generated 1497 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:30:40.805115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa41c42-2981-472a-a91b-de4e1075111e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:40.885469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975ffe13-9be1-4ce1-bcb4-e5cef69b27f7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:40.921896 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:40.929613 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787de836-9fbf-4e0e-88ff-c434088ce71a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:40.930613 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787de836-9fbf-4e0e-88ff-c434088ce71a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:40.931089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787de836-9fbf-4e0e-88ff-c434088ce71a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:40.931566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787de836-9fbf-4e0e-88ff-c434088ce71a None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:40.932506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787de836-9fbf-4e0e-88ff-c434088ce71a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:40.933041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787de836-9fbf-4e0e-88ff-c434088ce71a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:40.934716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787de836-9fbf-4e0e-88ff-c434088ce71a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:40.947076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa41c42-2981-472a-a91b-de4e1075111e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:40.955800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 470/952] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:40.969839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787de836-9fbf-4e0e-88ff-c434088ce71a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:40.999746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c800e8d-6d4a-46b6-bc4c-be14611501c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:41.002470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c800e8d-6d4a-46b6-bc4c-be14611501c7 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:41.002470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c800e8d-6d4a-46b6-bc4c-be14611501c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:41.002470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c800e8d-6d4a-46b6-bc4c-be14611501c7 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:41.002470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c800e8d-6d4a-46b6-bc4c-be14611501c7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:41.002470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c800e8d-6d4a-46b6-bc4c-be14611501c7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:41.003984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c800e8d-6d4a-46b6-bc4c-be14611501c7 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:41.040940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 483/953] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:41.125151 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787de836-9fbf-4e0e-88ff-c434088ce71a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:41.170134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83cb563-dd5f-4390-891f-85cb97ccc4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:41.171232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83cb563-dd5f-4390-891f-85cb97ccc4f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:41.171677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83cb563-dd5f-4390-891f-85cb97ccc4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:41.172183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83cb563-dd5f-4390-891f-85cb97ccc4f2 None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/7da74f16cb9b4b15a902947103a25cf6/roles/a2aaf3bf584d4607946a010016852567` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:41.177493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83cb563-dd5f-4390-891f-85cb97ccc4f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=7da74f16cb9b4b15a902947103a25cf6, role_id=a2aaf3bf584d4607946a010016852567)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:41.179158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83cb563-dd5f-4390-891f-85cb97ccc4f2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:41.212715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 471/954] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:30:40 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/7da74f16cb9b4b15a902947103a25cf6/roles/a2aaf3bf584d4607946a010016852567 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:30:42.438262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3bfc0-bd61-472a-89f2-aac601eb848b None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:42.438804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3bfc0-bd61-472a-89f2-aac601eb848b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:42.438804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3bfc0-bd61-472a-89f2-aac601eb848b None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:42.440497 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 484/955] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:42.448795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adea409-fff6-4836-b090-538e299ace76 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:42.448795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adea409-fff6-4836-b090-538e299ace76 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:42.448795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adea409-fff6-4836-b090-538e299ace76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:42.476905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5adea409-fff6-4836-b090-538e299ace76 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:42.478358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5adea409-fff6-4836-b090-538e299ace76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:42.539323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:42.568698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 472/956] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:42 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:42.916273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3bfc0-bd61-472a-89f2-aac601eb848b None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:42.932042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:43.049574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3bfc0-bd61-472a-89f2-aac601eb848b None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:43.090286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412621fc-c5ab-4b8b-801e-0b07ecb5b68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:43.091266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412621fc-c5ab-4b8b-801e-0b07ecb5b68e None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:43.091743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412621fc-c5ab-4b8b-801e-0b07ecb5b68e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:43.092220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412621fc-c5ab-4b8b-801e-0b07ecb5b68e None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:43.093115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412621fc-c5ab-4b8b-801e-0b07ecb5b68e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:43.094781 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412621fc-c5ab-4b8b-801e-0b07ecb5b68e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:43.116786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 485/957] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:30:42 2022] POST /identity/v3/services => generated 242 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:43.426197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5163f5b-7027-4abe-bb2e-a9bacc5d0726 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:43.426197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5163f5b-7027-4abe-bb2e-a9bacc5d0726 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:43.426197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5163f5b-7027-4abe-bb2e-a9bacc5d0726 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:43.427150 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 473/958] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:43.432510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb96e4-a00e-48be-98a4-4626b1de3452 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:43.432830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb96e4-a00e-48be-98a4-4626b1de3452 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:43.433099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eeb96e4-a00e-48be-98a4-4626b1de3452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:43.455257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7eeb96e4-a00e-48be-98a4-4626b1de3452 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:43.457229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7eeb96e4-a00e-48be-98a4-4626b1de3452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:43.533856 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:43.551324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 486/959] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:43.724063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06696535-bd4a-41fe-bb00-c8960cd23af1 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:43.724550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06696535-bd4a-41fe-bb00-c8960cd23af1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:43.724550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06696535-bd4a-41fe-bb00-c8960cd23af1 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:43.726036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 474/960] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:43.734495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca828b0a-3faf-4de8-87a9-8a67e0d26efd None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:43.734495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca828b0a-3faf-4de8-87a9-8a67e0d26efd None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:43.734495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca828b0a-3faf-4de8-87a9-8a67e0d26efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:43.765493 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-ca828b0a-3faf-4de8-87a9-8a67e0d26efd None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:43.766723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ca828b0a-3faf-4de8-87a9-8a67e0d26efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:43.846286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 487/961] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:43.876070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06696535-bd4a-41fe-bb00-c8960cd23af1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:43.892791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05d22b-c79f-4385-aa34-3109512dc5e7 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:43.893475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05d22b-c79f-4385-aa34-3109512dc5e7 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:43.894464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd05d22b-c79f-4385-aa34-3109512dc5e7 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:43.898644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 488/962] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:43 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:43.904097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa028b1-65ea-4d12-86ff-d959b160fb56 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:43.904378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa028b1-65ea-4d12-86ff-d959b160fb56 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:43.904741 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa028b1-65ea-4d12-86ff-d959b160fb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:43.927239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-6fa028b1-65ea-4d12-86ff-d959b160fb56 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:43.931337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6fa028b1-65ea-4d12-86ff-d959b160fb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:44.051904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 489/963] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:43 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:44.060208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06696535-bd4a-41fe-bb00-c8960cd23af1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:44.100806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661edfe-eb8a-4690-8f6b-4de610b67118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:44.100806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661edfe-eb8a-4690-8f6b-4de610b67118 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:44.100806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661edfe-eb8a-4690-8f6b-4de610b67118 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:44.100806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661edfe-eb8a-4690-8f6b-4de610b67118 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:44.101262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661edfe-eb8a-4690-8f6b-4de610b67118 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:44.101262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661edfe-eb8a-4690-8f6b-4de610b67118 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-6661edfe-eb8a-4690-8f6b-4de610b67118 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:44.110238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:44.111404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:44.112316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:44.113657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 475/964] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Sep 6 22:30:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:44.118342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa028b1-65ea-4d12-86ff-d959b160fb56 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:44.163055 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661edfe-eb8a-4690-8f6b-4de610b67118 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:44.203884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:44.221603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:44.293148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa028b1-65ea-4d12-86ff-d959b160fb56 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:44.299580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661edfe-eb8a-4690-8f6b-4de610b67118 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:44.337799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449c7b7-3af5-4db3-a930-cd41c997d174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:44.338444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449c7b7-3af5-4db3-a930-cd41c997d174 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:44.339332 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449c7b7-3af5-4db3-a930-cd41c997d174 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:44.339703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449c7b7-3af5-4db3-a930-cd41c997d174 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:44.340407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449c7b7-3af5-4db3-a930-cd41c997d174 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:44.340827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449c7b7-3af5-4db3-a930-cd41c997d174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:44.342646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8449c7b7-3af5-4db3-a930-cd41c997d174 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:44.347303 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314e1bf-634f-4834-850f-46c8954133c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:44.347920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314e1bf-634f-4834-850f-46c8954133c4 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:44.356830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314e1bf-634f-4834-850f-46c8954133c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:44.356830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314e1bf-634f-4834-850f-46c8954133c4 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:44.356830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314e1bf-634f-4834-850f-46c8954133c4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:44.356830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314e1bf-634f-4834-850f-46c8954133c4 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:44.356830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: WARNING keystone.server.flask.common [None req-9314e1bf-634f-4834-850f-46c8954133c4 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 06 22:30:44.357280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 490/965] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Sep 6 22:30:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:44.366002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449c7b7-3af5-4db3-a930-cd41c997d174 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:44.373468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.sql.core [None req-9314e1bf-634f-4834-850f-46c8954133c4 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 22:30:44.373468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 22:30:44.373468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [parameters: {'user_id': 'c2b16a654a7440ee8b09ed1788ddd54f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 22:30:44.373468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81665) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-9314e1bf-634f-4834-850f-46c8954133c4 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application conn.query(q) Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result.read() Sep 06 22:30:44.381517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 22:30:44.382401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application next(self.gen) Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application yield resource Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application next(self.gen) Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.session.rollback() Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.force_reraise() Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise self.value Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application session.commit() Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 06 22:30:44.383611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._prepare_impl() Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.session.flush() Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._flush(objects) Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application compat.raise_( Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application flush_context.execute() Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rec.execute(self) Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 06 22:30:44.384498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 06 22:30:44.385429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result = self._query(query) Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application conn.query(q) Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application result.read() Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 06 22:30:44.386297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [parameters: {'user_id': 'c2b16a654a7440ee8b09ed1788ddd54f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:44.387135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 06 22:30:44.387919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 06 22:30:44.388860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 06 22:30:44.388860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 06 22:30:44.388860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 06 22:30:44.388860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 06 22:30:44.388860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 06 22:30:44.388860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:44.388860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 476/966] 10.222.0.39 () {60 vars in 1131 bytes} [Tue Sep 6 22:30:44 2022] POST /identity/v3/users => generated 160 bytes in 221 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:44.388860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314e1bf-634f-4834-850f-46c8954133c4 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:44.502654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449c7b7-3af5-4db3-a930-cd41c997d174 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:44.523554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314e1bf-634f-4834-850f-46c8954133c4 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:44.526414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:44.538278 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:44.552891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bcb8c-9972-4a7d-9f1b-87e0202dded4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:44.553836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bcb8c-9972-4a7d-9f1b-87e0202dded4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:44.554365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bcb8c-9972-4a7d-9f1b-87e0202dded4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:44.554878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849bcb8c-9972-4a7d-9f1b-87e0202dded4 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:44.555523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bcb8c-9972-4a7d-9f1b-87e0202dded4 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:44.556144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bcb8c-9972-4a7d-9f1b-87e0202dded4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:44.557619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849bcb8c-9972-4a7d-9f1b-87e0202dded4 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:44.571906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 491/967] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Sep 6 22:30:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:44.573223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bcb8c-9972-4a7d-9f1b-87e0202dded4 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:44.586610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd5e5c-829f-40a3-a031-ce4c87702ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:44.587262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd5e5c-829f-40a3-a031-ce4c87702ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:44.587543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd5e5c-829f-40a3-a031-ce4c87702ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:44.587866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd5e5c-829f-40a3-a031-ce4c87702ce7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:44.602339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd5e5c-829f-40a3-a031-ce4c87702ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e5cd5e5c-829f-40a3-a031-ce4c87702ce7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:44.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:44.608445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:44.609333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 477/968] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Sep 6 22:30:44 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:44.610042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd5e5c-829f-40a3-a031-ce4c87702ce7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:44.708889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849bcb8c-9972-4a7d-9f1b-87e0202dded4 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:44.744389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd5e5c-829f-40a3-a031-ce4c87702ce7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:44.749045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895d640-55cd-4320-bacf-57d724678bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:44.749906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895d640-55cd-4320-bacf-57d724678bba None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:44.750388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895d640-55cd-4320-bacf-57d724678bba None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:44.750875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3895d640-55cd-4320-bacf-57d724678bba None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:44.760819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895d640-55cd-4320-bacf-57d724678bba None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:44.762554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3895d640-55cd-4320-bacf-57d724678bba None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-3895d640-55cd-4320-bacf-57d724678bba None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:44.778622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:44.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:44.780554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:44.780554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:44.780554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:44.780554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:44.780554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:44.780554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:44.780554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:44.780554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:44.780554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 06 22:30:44.780554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:44.781991 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 492/969] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:30:44 2022] GET /identity/v3/users/cinder => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:44.782778 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895d640-55cd-4320-bacf-57d724678bba None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:44.792484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501f6de-993e-4c33-ba44-62f98cabb022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:44.793493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f6de-993e-4c33-ba44-62f98cabb022 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:44.793770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f6de-993e-4c33-ba44-62f98cabb022 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:44.794608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5501f6de-993e-4c33-ba44-62f98cabb022 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:44.795282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501f6de-993e-4c33-ba44-62f98cabb022 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:44.796894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5501f6de-993e-4c33-ba44-62f98cabb022 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:44.837001 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 478/970] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:30:44 2022] POST /identity/v3/endpoints => generated 349 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:44.838088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501f6de-993e-4c33-ba44-62f98cabb022 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:44.844138 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:44.898686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:44.932872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3895d640-55cd-4320-bacf-57d724678bba None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:44.993147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63f14e-93c9-46bd-a5ac-347789dddc8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:44.994291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63f14e-93c9-46bd-a5ac-347789dddc8f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:44.994766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63f14e-93c9-46bd-a5ac-347789dddc8f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:44.995230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63f14e-93c9-46bd-a5ac-347789dddc8f None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:44.996125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63f14e-93c9-46bd-a5ac-347789dddc8f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:44.998641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63f14e-93c9-46bd-a5ac-347789dddc8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:45.000501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63f14e-93c9-46bd-a5ac-347789dddc8f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:45.012207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5501f6de-993e-4c33-ba44-62f98cabb022 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:45.021877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 493/971] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Sep 6 22:30:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:45.026224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63f14e-93c9-46bd-a5ac-347789dddc8f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:45.072294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80d6a2-e9a2-4a6c-92bd-54156900f43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:45.072950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80d6a2-e9a2-4a6c-92bd-54156900f43b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:45.073229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80d6a2-e9a2-4a6c-92bd-54156900f43b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:45.073500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af80d6a2-e9a2-4a6c-92bd-54156900f43b None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:45.074204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80d6a2-e9a2-4a6c-92bd-54156900f43b None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:45.074506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80d6a2-e9a2-4a6c-92bd-54156900f43b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:45.075929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af80d6a2-e9a2-4a6c-92bd-54156900f43b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:45.129037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 479/972] 10.222.0.39 () {56 vars in 1101 bytes} [Tue Sep 6 22:30:44 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:45.162575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:45.167417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63f14e-93c9-46bd-a5ac-347789dddc8f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:45.225141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb9e04-60ae-483b-9d34-85bc03a0a173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:45.226205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb9e04-60ae-483b-9d34-85bc03a0a173 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:45.226691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb9e04-60ae-483b-9d34-85bc03a0a173 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:45.227165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acb9e04-60ae-483b-9d34-85bc03a0a173 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:45.239592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb9e04-60ae-483b-9d34-85bc03a0a173 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:45.241469 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acb9e04-60ae-483b-9d34-85bc03a0a173 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-7acb9e04-60ae-483b-9d34-85bc03a0a173 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:45.262568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:45.263602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:45.264549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:45.264549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:45.264549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:45.266835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 494/973] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:45.272977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80d6a2-e9a2-4a6c-92bd-54156900f43b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:45.368716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:45.397733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af80d6a2-e9a2-4a6c-92bd-54156900f43b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:45.449495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2143d-ef45-4f26-a4a2-32e97eecda28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:45.450094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2143d-ef45-4f26-a4a2-32e97eecda28 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:45.450370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2143d-ef45-4f26-a4a2-32e97eecda28 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:45.450641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2143d-ef45-4f26-a4a2-32e97eecda28 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:45.451291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2143d-ef45-4f26-a4a2-32e97eecda28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:45.451592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2143d-ef45-4f26-a4a2-32e97eecda28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:45.453018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc2143d-ef45-4f26-a4a2-32e97eecda28 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:45.472650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 480/974] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:45.479890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb9e04-60ae-483b-9d34-85bc03a0a173 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:45.607161 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b344d1-62c1-4773-bb57-a2878d98d769 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:45.607443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b344d1-62c1-4773-bb57-a2878d98d769 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:45.607716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b344d1-62c1-4773-bb57-a2878d98d769 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:45.609342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 481/975] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:45.615734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec06374-2415-455c-9832-3eb663ab8de2 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:45.616018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec06374-2415-455c-9832-3eb663ab8de2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:45.616293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec06374-2415-455c-9832-3eb663ab8de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:45.618396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acb9e04-60ae-483b-9d34-85bc03a0a173 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:45.640666 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6ec06374-2415-455c-9832-3eb663ab8de2 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:45.642299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6ec06374-2415-455c-9832-3eb663ab8de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:45.661828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b89e9-aca5-4b96-a25d-b699df799199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:45.662648 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b89e9-aca5-4b96-a25d-b699df799199 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:45.662994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b89e9-aca5-4b96-a25d-b699df799199 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:45.663599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741b89e9-aca5-4b96-a25d-b699df799199 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:45.674106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b89e9-aca5-4b96-a25d-b699df799199 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:45.676110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741b89e9-aca5-4b96-a25d-b699df799199 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-741b89e9-aca5-4b96-a25d-b699df799199 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:45.689421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:45.690967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:45.693194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:45.693194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:45.693194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:45.693194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:45.693194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:45.693194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:45.693194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:45.693194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:45.693194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 06 22:30:45.693194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:45.695837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 495/976] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Sep 6 22:30:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:45.697341 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b89e9-aca5-4b96-a25d-b699df799199 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:45.713572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:45.743209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 482/977] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:45 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:45.814123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741b89e9-aca5-4b96-a25d-b699df799199 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:45.851964 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3241d0-9e9d-473e-8029-0201319f88ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:45.852875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3241d0-9e9d-473e-8029-0201319f88ef None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:45.853297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3241d0-9e9d-473e-8029-0201319f88ef None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:45.853728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3241d0-9e9d-473e-8029-0201319f88ef None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:45.854592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3241d0-9e9d-473e-8029-0201319f88ef None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:45.855037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3241d0-9e9d-473e-8029-0201319f88ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:45.856799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3241d0-9e9d-473e-8029-0201319f88ef None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:45.878906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 496/978] 10.222.0.39 () {56 vars in 1149 bytes} [Tue Sep 6 22:30:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:45.888514 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec06374-2415-455c-9832-3eb663ab8de2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:45.970475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:46.005951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec06374-2415-455c-9832-3eb663ab8de2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:46.020621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3241d0-9e9d-473e-8029-0201319f88ef None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:46.052597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84335ad2-b7ab-4773-b852-59bb55e88c4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:46.053490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84335ad2-b7ab-4773-b852-59bb55e88c4d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:46.053770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84335ad2-b7ab-4773-b852-59bb55e88c4d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:46.054038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84335ad2-b7ab-4773-b852-59bb55e88c4d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:46.109037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:46.139933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84335ad2-b7ab-4773-b852-59bb55e88c4d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:46.142378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84335ad2-b7ab-4773-b852-59bb55e88c4d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:46.160779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3241d0-9e9d-473e-8029-0201319f88ef None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:46.206093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668de5c-47b2-479e-b1b2-5360b7684172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:46.207256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668de5c-47b2-479e-b1b2-5360b7684172 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:46.207509 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668de5c-47b2-479e-b1b2-5360b7684172 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:46.207797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a668de5c-47b2-479e-b1b2-5360b7684172 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:46.208357 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668de5c-47b2-479e-b1b2-5360b7684172 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:46.209753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a668de5c-47b2-479e-b1b2-5360b7684172 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-a668de5c-47b2-479e-b1b2-5360b7684172 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:46.218671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:46.219883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 22:30:46.220993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:46.224793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 497/979] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:46 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:46.225715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668de5c-47b2-479e-b1b2-5360b7684172 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:46.237174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:46.281922 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 483/980] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:45 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:30:46.282966 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84335ad2-b7ab-4773-b852-59bb55e88c4d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:46.389571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a668de5c-47b2-479e-b1b2-5360b7684172 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:46.433181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:46.441168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088d535-b9fc-4b2d-9d7e-fcf8831e4195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:46.442047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088d535-b9fc-4b2d-9d7e-fcf8831e4195 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:46.442548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088d535-b9fc-4b2d-9d7e-fcf8831e4195 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:46.443037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2088d535-b9fc-4b2d-9d7e-fcf8831e4195 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:46.443964 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088d535-b9fc-4b2d-9d7e-fcf8831e4195 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:46.444684 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088d535-b9fc-4b2d-9d7e-fcf8831e4195 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:46.445800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2088d535-b9fc-4b2d-9d7e-fcf8831e4195 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:46.450231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84335ad2-b7ab-4773-b852-59bb55e88c4d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:46.451671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 498/981] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:46 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:46.457833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088d535-b9fc-4b2d-9d7e-fcf8831e4195 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:46.492632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e208de-110f-46c3-bb53-a5651aa7915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:46.493576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e208de-110f-46c3-bb53-a5651aa7915a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:46.493875 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e208de-110f-46c3-bb53-a5651aa7915a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:46.494355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e208de-110f-46c3-bb53-a5651aa7915a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:46.510315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e208de-110f-46c3-bb53-a5651aa7915a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:46.511745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e208de-110f-46c3-bb53-a5651aa7915a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-36e208de-110f-46c3-bb53-a5651aa7915a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:46.528858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:46.529825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:46.530702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:46.530702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:46.530702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:46.536126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 484/982] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:46.537079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e208de-110f-46c3-bb53-a5651aa7915a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:46.581249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:46.597594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2088d535-b9fc-4b2d-9d7e-fcf8831e4195 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:46.643482 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418058e-904a-49f7-b98a-0315345d4113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:46.645447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418058e-904a-49f7-b98a-0315345d4113 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:46.645757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418058e-904a-49f7-b98a-0315345d4113 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:46.645998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b418058e-904a-49f7-b98a-0315345d4113 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:46.646621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418058e-904a-49f7-b98a-0315345d4113 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:46.646923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418058e-904a-49f7-b98a-0315345d4113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:46.648295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b418058e-904a-49f7-b98a-0315345d4113 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:46.658557 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 499/983] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:46 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:46.660278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418058e-904a-49f7-b98a-0315345d4113 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:46.686338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e208de-110f-46c3-bb53-a5651aa7915a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:46.724601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdca438-90f3-46ce-a683-41d6e99269a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:46.727779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdca438-90f3-46ce-a683-41d6e99269a1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:46.728279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdca438-90f3-46ce-a683-41d6e99269a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:46.728828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdca438-90f3-46ce-a683-41d6e99269a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:46.729687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdca438-90f3-46ce-a683-41d6e99269a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:46.730190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdca438-90f3-46ce-a683-41d6e99269a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:46.731579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:46.732008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdca438-90f3-46ce-a683-41d6e99269a1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:46.745715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 485/984] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:46.751382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdca438-90f3-46ce-a683-41d6e99269a1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:46.786618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b418058e-904a-49f7-b98a-0315345d4113 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:46.819578 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9ba4a-fde1-4423-b4af-299fb4b5d677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:46.820182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9ba4a-fde1-4423-b4af-299fb4b5d677 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:46.820462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9ba4a-fde1-4423-b4af-299fb4b5d677 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:46.820743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee9ba4a-fde1-4423-b4af-299fb4b5d677 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:46.821464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9ba4a-fde1-4423-b4af-299fb4b5d677 None admin] RBAC: Adding query filter params (interface=public, service_id=de892c5fedf2451a9ff8e48a9f0c90b8, region_id=RegionOne) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:46.821770 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9ba4a-fde1-4423-b4af-299fb4b5d677 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:46.823201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee9ba4a-fde1-4423-b4af-299fb4b5d677 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:46.833135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 500/985] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Sep 6 22:30:46 2022] GET /identity/v3/endpoints?service_id=de892c5fedf2451a9ff8e48a9f0c90b8&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:46.834505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9ba4a-fde1-4423-b4af-299fb4b5d677 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:46.848587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:46.870728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdca438-90f3-46ce-a683-41d6e99269a1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:46.912799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d4e69-2d7d-47e4-8b73-4e2bd691f964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:46.914246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d4e69-2d7d-47e4-8b73-4e2bd691f964 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:46.914747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d4e69-2d7d-47e4-8b73-4e2bd691f964 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:46.915211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6d4e69-2d7d-47e4-8b73-4e2bd691f964 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:46.925162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d4e69-2d7d-47e4-8b73-4e2bd691f964 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:46.927415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6d4e69-2d7d-47e4-8b73-4e2bd691f964 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-7d6d4e69-2d7d-47e4-8b73-4e2bd691f964 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:46.941353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:46.942639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:46.943553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:46.943553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:46.943553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:46.943553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:46.943553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:46.943553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:46.943553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:46.943553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:46.951402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 486/986] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:46.952527 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d4e69-2d7d-47e4-8b73-4e2bd691f964 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:46.961770 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee9ba4a-fde1-4423-b4af-299fb4b5d677 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:46.990279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:47.001712 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247d14b-93f9-4656-9531-6a7b85fc054d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:47.002144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247d14b-93f9-4656-9531-6a7b85fc054d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:47.002424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247d14b-93f9-4656-9531-6a7b85fc054d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:47.002694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247d14b-93f9-4656-9531-6a7b85fc054d None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:47.003636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247d14b-93f9-4656-9531-6a7b85fc054d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:47.004883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247d14b-93f9-4656-9531-6a7b85fc054d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:47.021347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 501/987] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Sep 6 22:30:46 2022] GET /identity/v3/services => generated 1727 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:30:47.080324 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6d4e69-2d7d-47e4-8b73-4e2bd691f964 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:47.124063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd53dba-748c-4ed1-959e-bd501faa2353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:47.125101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd53dba-748c-4ed1-959e-bd501faa2353 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:47.125573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd53dba-748c-4ed1-959e-bd501faa2353 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:47.126048 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd53dba-748c-4ed1-959e-bd501faa2353 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:47.127113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd53dba-748c-4ed1-959e-bd501faa2353 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:47.127613 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd53dba-748c-4ed1-959e-bd501faa2353 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:47.129367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd53dba-748c-4ed1-959e-bd501faa2353 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:47.153599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 487/988] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:47.156415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247d14b-93f9-4656-9531-6a7b85fc054d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:47.191955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c704af-5f66-4334-ace1-551b0f16402c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:47.192281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c704af-5f66-4334-ace1-551b0f16402c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:47.192494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c704af-5f66-4334-ace1-551b0f16402c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:47.194148 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 488/989] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:30:47.201589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199e3ff-4fe3-4bb1-bb32-1da1a0aa1607 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:47.202502 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199e3ff-4fe3-4bb1-bb32-1da1a0aa1607 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:47.202943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199e3ff-4fe3-4bb1-bb32-1da1a0aa1607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:47.210687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:47.231883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c199e3ff-4fe3-4bb1-bb32-1da1a0aa1607 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:47.235612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c199e3ff-4fe3-4bb1-bb32-1da1a0aa1607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:47.294384 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247d14b-93f9-4656-9531-6a7b85fc054d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:47.335678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9e5c4-254c-482b-a722-14c49cdf1e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:47.336611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9e5c4-254c-482b-a722-14c49cdf1e4c None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:47.336943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9e5c4-254c-482b-a722-14c49cdf1e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:47.337499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9e5c4-254c-482b-a722-14c49cdf1e4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:47.338238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9e5c4-254c-482b-a722-14c49cdf1e4c None admin] RBAC: Adding query filter params (role.id=a2aaf3bf584d4607946a010016852567, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=7da74f16cb9b4b15a902947103a25cf6) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:47.338547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9e5c4-254c-482b-a722-14c49cdf1e4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:47.340589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9e5c4-254c-482b-a722-14c49cdf1e4c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:47.347586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 502/990] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:47 2022] GET /identity/v3/role_assignments?user.id=7da74f16cb9b4b15a902947103a25cf6&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=a2aaf3bf584d4607946a010016852567 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:47.372891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 489/991] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:47 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:47.645790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9e5c4-254c-482b-a722-14c49cdf1e4c None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:47.742959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:47.800785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9e5c4-254c-482b-a722-14c49cdf1e4c None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:47.848970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac242609-d9f3-4e68-8456-29094797ca82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:47.853702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac242609-d9f3-4e68-8456-29094797ca82 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:47.855364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac242609-d9f3-4e68-8456-29094797ca82 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:47.857132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac242609-d9f3-4e68-8456-29094797ca82 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:47.861061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac242609-d9f3-4e68-8456-29094797ca82 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:47.862951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac242609-d9f3-4e68-8456-29094797ca82 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-ac242609-d9f3-4e68-8456-29094797ca82 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:47.873261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:47.874273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:47.875146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:47.880785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199e3ff-4fe3-4bb1-bb32-1da1a0aa1607 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:47.880920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 503/992] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Sep 6 22:30:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:48.017827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199e3ff-4fe3-4bb1-bb32-1da1a0aa1607 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:48.047237 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:48.053702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a6726-0093-48cd-8e4c-2906527ae3a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:48.054315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6726-0093-48cd-8e4c-2906527ae3a6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:48.054595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6726-0093-48cd-8e4c-2906527ae3a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:48.054870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a6726-0093-48cd-8e4c-2906527ae3a6 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:48.055500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a6726-0093-48cd-8e4c-2906527ae3a6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:48.055804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a6726-0093-48cd-8e4c-2906527ae3a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:48.057207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a6726-0093-48cd-8e4c-2906527ae3a6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:48.062368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 490/993] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Sep 6 22:30:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:48.068530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac242609-d9f3-4e68-8456-29094797ca82 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:48.169985 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac242609-d9f3-4e68-8456-29094797ca82 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:48.186371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:48.204562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb0af7-e930-4aff-a959-b63714ceb1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:48.205245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb0af7-e930-4aff-a959-b63714ceb1b8 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:48.205527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb0af7-e930-4aff-a959-b63714ceb1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:48.205801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcb0af7-e930-4aff-a959-b63714ceb1b8 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:48.206531 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb0af7-e930-4aff-a959-b63714ceb1b8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:48.206832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb0af7-e930-4aff-a959-b63714ceb1b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:48.208190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcb0af7-e930-4aff-a959-b63714ceb1b8 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:48.213311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 504/994] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Sep 6 22:30:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:48.218938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a6726-0093-48cd-8e4c-2906527ae3a6 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:48.323432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a6726-0093-48cd-8e4c-2906527ae3a6 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:48.357688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ffe66-2d3c-4824-bcbf-aaa382a571ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:48.358296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ffe66-2d3c-4824-bcbf-aaa382a571ee None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:48.358574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ffe66-2d3c-4824-bcbf-aaa382a571ee None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:48.358846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ffe66-2d3c-4824-bcbf-aaa382a571ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:48.359547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ffe66-2d3c-4824-bcbf-aaa382a571ee None admin] RBAC: Adding query filter params (interface=admin, service_id=d2ae420d07934b5eb08a96c65b69c8e6, region_id=RegionOne) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:48.359853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ffe66-2d3c-4824-bcbf-aaa382a571ee None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:48.361257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ffe66-2d3c-4824-bcbf-aaa382a571ee None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:48.371788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 491/995] 10.222.0.39 () {56 vars in 1249 bytes} [Tue Sep 6 22:30:48 2022] GET /identity/v3/endpoints?service_id=d2ae420d07934b5eb08a96c65b69c8e6&interface=admin®ion_id=RegionOne => generated 193 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:48.373405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb0af7-e930-4aff-a959-b63714ceb1b8 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:48.473427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcb0af7-e930-4aff-a959-b63714ceb1b8 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:48.505777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9bec91-9188-4c89-9752-36a5c9a15fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:48.506365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bec91-9188-4c89-9752-36a5c9a15fee None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:48.506637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bec91-9188-4c89-9752-36a5c9a15fee None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:48.506899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9bec91-9188-4c89-9752-36a5c9a15fee None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:48.507508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bec91-9188-4c89-9752-36a5c9a15fee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:48.509096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9bec91-9188-4c89-9752-36a5c9a15fee None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:48.515322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 505/996] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Sep 6 22:30:48 2022] GET /identity/v3/services => generated 1727 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:30:49.225942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e08b5-9e2e-46f1-a74e-4173b3c2b76f None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:49.226471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e08b5-9e2e-46f1-a74e-4173b3c2b76f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:49.226471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e08b5-9e2e-46f1-a74e-4173b3c2b76f None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:49.228245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 492/997] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:49.233889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18667637-d8ed-4b8d-a125-fd18b8308c52 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:49.234322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18667637-d8ed-4b8d-a125-fd18b8308c52 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:49.234660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18667637-d8ed-4b8d-a125-fd18b8308c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:49.253824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-18667637-d8ed-4b8d-a125-fd18b8308c52 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:49.255028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-18667637-d8ed-4b8d-a125-fd18b8308c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:49.296947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:49.324960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 506/998] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:49.598545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e08b5-9e2e-46f1-a74e-4173b3c2b76f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:49.609147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477d808-ec8a-429f-a02f-f368269ec9ca None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:49.609428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477d808-ec8a-429f-a02f-f368269ec9ca None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:49.609697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477d808-ec8a-429f-a02f-f368269ec9ca None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:49.611162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 507/999] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:30:49.616002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fd141-363e-465d-accf-870db1097182 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:49.616292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fd141-363e-465d-accf-870db1097182 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:49.616646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21fd141-363e-465d-accf-870db1097182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:49.636070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-b21fd141-363e-465d-accf-870db1097182 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:49.637420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b21fd141-363e-465d-accf-870db1097182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:49.662730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:49.706851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 508/1000] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:49 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:49.719864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e08b5-9e2e-46f1-a74e-4173b3c2b76f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:49.758878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722b9e4-aa81-4703-964f-0adce12992ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:49.759513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722b9e4-aa81-4703-964f-0adce12992ca None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:49.759784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722b9e4-aa81-4703-964f-0adce12992ca None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:49.760050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9722b9e4-aa81-4703-964f-0adce12992ca None admin] PATH_INFO: `/v3/services/compute` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:49.760691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722b9e4-aa81-4703-964f-0adce12992ca None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:49.762135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9722b9e4-aa81-4703-964f-0adce12992ca None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-9722b9e4-aa81-4703-964f-0adce12992ca None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:49.777269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:49.778323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:49.779865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 493/1001] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:49 2022] GET /identity/v3/services/compute => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:49.784524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fd141-363e-465d-accf-870db1097182 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:49.885715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21fd141-363e-465d-accf-870db1097182 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:49.888276 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:49.921196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118a90f-6905-436a-9864-87509af49e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:49.922012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118a90f-6905-436a-9864-87509af49e8b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:49.922443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118a90f-6905-436a-9864-87509af49e8b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:49.922853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2118a90f-6905-436a-9864-87509af49e8b None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:49.923623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118a90f-6905-436a-9864-87509af49e8b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:49.924061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118a90f-6905-436a-9864-87509af49e8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:49.926098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2118a90f-6905-436a-9864-87509af49e8b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:49.935433 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 509/1002] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:49.936988 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722b9e4-aa81-4703-964f-0adce12992ca None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:49.986981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118a90f-6905-436a-9864-87509af49e8b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:50.051664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9722b9e4-aa81-4703-964f-0adce12992ca None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:50.078035 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:50.091038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d979ad0-c9b1-4dfd-be42-c35597c13a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:50.091759 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d979ad0-c9b1-4dfd-be42-c35597c13a3e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:50.092047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d979ad0-c9b1-4dfd-be42-c35597c13a3e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:50.092320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d979ad0-c9b1-4dfd-be42-c35597c13a3e None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:50.093099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d979ad0-c9b1-4dfd-be42-c35597c13a3e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:50.093402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d979ad0-c9b1-4dfd-be42-c35597c13a3e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:50.094863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d979ad0-c9b1-4dfd-be42-c35597c13a3e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:50.099671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2118a90f-6905-436a-9864-87509af49e8b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:50.105044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 494/1003] 10.222.0.39 () {56 vars in 1112 bytes} [Tue Sep 6 22:30:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:50.105978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d979ad0-c9b1-4dfd-be42-c35597c13a3e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:50.139932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f46f2-1ae8-4cf5-bbcd-dbb015f13ff3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:50.140530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f46f2-1ae8-4cf5-bbcd-dbb015f13ff3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:50.140847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f46f2-1ae8-4cf5-bbcd-dbb015f13ff3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:50.141122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f46f2-1ae8-4cf5-bbcd-dbb015f13ff3 None admin] PATH_INFO: `/v3/services/image` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:50.141706 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f46f2-1ae8-4cf5-bbcd-dbb015f13ff3 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:50.143056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f46f2-1ae8-4cf5-bbcd-dbb015f13ff3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-8f4f46f2-1ae8-4cf5-bbcd-dbb015f13ff3 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:50.153546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:50.154607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:50.155466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 510/1004] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Sep 6 22:30:49 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:50.161325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f46f2-1ae8-4cf5-bbcd-dbb015f13ff3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:50.194978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:50.219911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d979ad0-c9b1-4dfd-be42-c35597c13a3e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:50.255601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b40c4-0c5e-4954-a424-ceaf9e88896e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:50.256254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b40c4-0c5e-4954-a424-ceaf9e88896e None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:50.256539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b40c4-0c5e-4954-a424-ceaf9e88896e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:50.256794 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530b40c4-0c5e-4954-a424-ceaf9e88896e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:50.257532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b40c4-0c5e-4954-a424-ceaf9e88896e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:50.259147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530b40c4-0c5e-4954-a424-ceaf9e88896e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:50.270069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f46f2-1ae8-4cf5-bbcd-dbb015f13ff3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:50.292970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 495/1005] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:30:50 2022] POST /identity/v3/endpoints => generated 334 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:50.319525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d20abaf-b53c-42db-a371-28dd40283d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:50.320151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20abaf-b53c-42db-a371-28dd40283d6e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:50.320430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20abaf-b53c-42db-a371-28dd40283d6e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:50.320700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d20abaf-b53c-42db-a371-28dd40283d6e None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:50.321373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20abaf-b53c-42db-a371-28dd40283d6e None admin] RBAC: Adding query filter params (name=image) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:50.321676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20abaf-b53c-42db-a371-28dd40283d6e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:50.323100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d20abaf-b53c-42db-a371-28dd40283d6e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:50.329070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 511/1006] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:50.335310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b40c4-0c5e-4954-a424-ceaf9e88896e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:50.386634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:50.449854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530b40c4-0c5e-4954-a424-ceaf9e88896e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:50.485935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be70be3-1c30-4cbc-bc82-ebbb12ee5a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:50.486548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be70be3-1c30-4cbc-bc82-ebbb12ee5a72 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:50.486835 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be70be3-1c30-4cbc-bc82-ebbb12ee5a72 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:50.487104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be70be3-1c30-4cbc-bc82-ebbb12ee5a72 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:50.487790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be70be3-1c30-4cbc-bc82-ebbb12ee5a72 None admin] RBAC: Adding query filter params (type=image) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:50.488093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be70be3-1c30-4cbc-bc82-ebbb12ee5a72 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:50.489500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be70be3-1c30-4cbc-bc82-ebbb12ee5a72 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:50.494483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 496/1007] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:50.703706 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c024a7-1d5f-4dcf-ae92-ec9f2bb1c19f None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:50.704020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c024a7-1d5f-4dcf-ae92-ec9f2bb1c19f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:50.704201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c024a7-1d5f-4dcf-ae92-ec9f2bb1c19f None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:50.705672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 512/1008] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:50.711563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951743-aaf8-41c5-9f99-e8a40dd4a10a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:50.711850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951743-aaf8-41c5-9f99-e8a40dd4a10a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:50.712123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d951743-aaf8-41c5-9f99-e8a40dd4a10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:50.731120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2d951743-aaf8-41c5-9f99-e8a40dd4a10a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:50.732299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2d951743-aaf8-41c5-9f99-e8a40dd4a10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:50.802166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 497/1009] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:50 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:51.067290 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c024a7-1d5f-4dcf-ae92-ec9f2bb1c19f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:51.161385 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:51.172042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c024a7-1d5f-4dcf-ae92-ec9f2bb1c19f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:51.205599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d48195-c054-4527-ac05-0c6ae2ceb4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:51.206896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d48195-c054-4527-ac05-0c6ae2ceb4dc None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:51.207246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d48195-c054-4527-ac05-0c6ae2ceb4dc None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:51.207423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d48195-c054-4527-ac05-0c6ae2ceb4dc None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:51.207989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d48195-c054-4527-ac05-0c6ae2ceb4dc None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:51.209362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d48195-c054-4527-ac05-0c6ae2ceb4dc None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-38d48195-c054-4527-ac05-0c6ae2ceb4dc None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:51.218191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:51.219210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 06 22:30:51.220088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:51.227311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 513/1010] 10.222.0.39 () {56 vars in 1126 bytes} [Tue Sep 6 22:30:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:51.227470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d951743-aaf8-41c5-9f99-e8a40dd4a10a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:51.269491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:51.336210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d951743-aaf8-41c5-9f99-e8a40dd4a10a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:51.371834 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1565e4-0a37-4be0-9923-4b3594e48ba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:51.372843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1565e4-0a37-4be0-9923-4b3594e48ba9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:51.373272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1565e4-0a37-4be0-9923-4b3594e48ba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:51.373691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1565e4-0a37-4be0-9923-4b3594e48ba9 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:51.374564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1565e4-0a37-4be0-9923-4b3594e48ba9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:51.375020 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1565e4-0a37-4be0-9923-4b3594e48ba9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:51.376604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1565e4-0a37-4be0-9923-4b3594e48ba9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:51.383762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 498/1011] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Sep 6 22:30:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:51.389275 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d48195-c054-4527-ac05-0c6ae2ceb4dc None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:51.495388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d48195-c054-4527-ac05-0c6ae2ceb4dc None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:51.531575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338840a5-f6d5-4278-b636-420b1e15e8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:51.532180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338840a5-f6d5-4278-b636-420b1e15e8d2 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:51.532458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338840a5-f6d5-4278-b636-420b1e15e8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:51.532728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338840a5-f6d5-4278-b636-420b1e15e8d2 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:51.533377 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338840a5-f6d5-4278-b636-420b1e15e8d2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:51.533678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338840a5-f6d5-4278-b636-420b1e15e8d2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:51.535159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338840a5-f6d5-4278-b636-420b1e15e8d2 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:51.540378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 514/1012] 10.222.0.39 () {56 vars in 1122 bytes} [Tue Sep 6 22:30:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:51.546210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1565e4-0a37-4be0-9923-4b3594e48ba9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:51.660235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1565e4-0a37-4be0-9923-4b3594e48ba9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:51.662940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:51.700453 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f85da8b-6c31-4221-8b89-ab71bedba63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:51.701093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85da8b-6c31-4221-8b89-ab71bedba63d None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:51.701370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85da8b-6c31-4221-8b89-ab71bedba63d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:51.701638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f85da8b-6c31-4221-8b89-ab71bedba63d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:51.702312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85da8b-6c31-4221-8b89-ab71bedba63d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:51.703699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f85da8b-6c31-4221-8b89-ab71bedba63d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:51.723707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 499/1013] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:30:51 2022] POST /identity/v3/endpoints => generated 325 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:52.556212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac737c4a-1ec0-4a2e-8d10-8a53fc5a81b2 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:52.556722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac737c4a-1ec0-4a2e-8d10-8a53fc5a81b2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:52.556722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac737c4a-1ec0-4a2e-8d10-8a53fc5a81b2 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:52.558238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 515/1014] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:52.563324 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73335-7b61-43ca-8ba3-eacb331081de None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:52.563717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73335-7b61-43ca-8ba3-eacb331081de None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:52.563883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b73335-7b61-43ca-8ba3-eacb331081de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:52.583823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-75b73335-7b61-43ca-8ba3-eacb331081de None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:52.585093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-75b73335-7b61-43ca-8ba3-eacb331081de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:52.665465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 500/1015] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:52 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:52.715542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb3769-4b53-4d74-8ad6-1eecd8ab02da None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:52.715835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb3769-4b53-4d74-8ad6-1eecd8ab02da None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:52.716098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafb3769-4b53-4d74-8ad6-1eecd8ab02da None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:52.717600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 516/1016] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:52.722690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6c09e-5c3f-439c-a616-878ec4f00302 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:52.722976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6c09e-5c3f-439c-a616-878ec4f00302 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:52.723246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc6c09e-5c3f-439c-a616-878ec4f00302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:52.741555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7cc6c09e-5c3f-439c-a616-878ec4f00302 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:52.742747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7cc6c09e-5c3f-439c-a616-878ec4f00302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:52.810493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 501/1017] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:52 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:52.934395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb3769-4b53-4d74-8ad6-1eecd8ab02da None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:52.958278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:53.040814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafb3769-4b53-4d74-8ad6-1eecd8ab02da None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:53.075105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3cfb8-c19e-49ed-bbab-c6d854921826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:53.075649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3cfb8-c19e-49ed-bbab-c6d854921826 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:53.075920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3cfb8-c19e-49ed-bbab-c6d854921826 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:53.076184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df3cfb8-c19e-49ed-bbab-c6d854921826 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:53.085291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6c09e-5c3f-439c-a616-878ec4f00302 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:53.086473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df3cfb8-c19e-49ed-bbab-c6d854921826 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-4df3cfb8-c19e-49ed-bbab-c6d854921826 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:53.091322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:53.092266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:30:53.093524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:53.097191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 517/1018] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Sep 6 22:30:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:53.098519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3cfb8-c19e-49ed-bbab-c6d854921826 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:53.106186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:53.184470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:53.191946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc6c09e-5c3f-439c-a616-878ec4f00302 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:53.204631 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df3cfb8-c19e-49ed-bbab-c6d854921826 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:53.226622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377d4e5-15b7-4699-9ce7-2fdd414d6906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:53.227224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377d4e5-15b7-4699-9ce7-2fdd414d6906 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:53.227503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377d4e5-15b7-4699-9ce7-2fdd414d6906 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:53.227768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377d4e5-15b7-4699-9ce7-2fdd414d6906 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:53.228414 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377d4e5-15b7-4699-9ce7-2fdd414d6906 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:53.229792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377d4e5-15b7-4699-9ce7-2fdd414d6906 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:53.242331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335e344-4c26-47d8-8497-f22e758131fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:53.242943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335e344-4c26-47d8-8497-f22e758131fa None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:53.243264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335e344-4c26-47d8-8497-f22e758131fa None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:53.246126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 502/1019] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:30:53 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:30:53.249214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3335e344-4c26-47d8-8497-f22e758131fa None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:53.250527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335e344-4c26-47d8-8497-f22e758131fa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:53.250843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335e344-4c26-47d8-8497-f22e758131fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:53.252384 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3335e344-4c26-47d8-8497-f22e758131fa None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:53.269412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 518/1020] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Sep 6 22:30:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:53.271210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377d4e5-15b7-4699-9ce7-2fdd414d6906 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:53.359850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:53.392069 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377d4e5-15b7-4699-9ce7-2fdd414d6906 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:53.428853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b7112-f9e7-4cb7-930e-d22594a32dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:53.429488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b7112-f9e7-4cb7-930e-d22594a32dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:53.429760 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b7112-f9e7-4cb7-930e-d22594a32dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:53.430026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b7112-f9e7-4cb7-930e-d22594a32dc5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:53.441465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b7112-f9e7-4cb7-930e-d22594a32dc5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:53.442953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b7112-f9e7-4cb7-930e-d22594a32dc5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-206b7112-f9e7-4cb7-930e-d22594a32dc5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:53.458524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:53.459565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:53.460688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:53.460688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:53.460688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:53.460688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 503/1021] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:53.465727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335e344-4c26-47d8-8497-f22e758131fa None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:53.580385 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3335e344-4c26-47d8-8497-f22e758131fa None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:53.616113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e2166-4752-4864-95bf-bb38077abfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:53.616637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e2166-4752-4864-95bf-bb38077abfef None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:53.616957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e2166-4752-4864-95bf-bb38077abfef None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:53.617230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88e2166-4752-4864-95bf-bb38077abfef None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:53.617865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e2166-4752-4864-95bf-bb38077abfef None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:53.618167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e2166-4752-4864-95bf-bb38077abfef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:53.619522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88e2166-4752-4864-95bf-bb38077abfef None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:53.636475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 519/1022] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:53.637824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b7112-f9e7-4cb7-930e-d22594a32dc5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:53.749847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b7112-f9e7-4cb7-930e-d22594a32dc5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:53.777519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:53.787836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8046a78-848d-42d2-a3eb-ab1f7836b5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:53.788493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046a78-848d-42d2-a3eb-ab1f7836b5c1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:53.788792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046a78-848d-42d2-a3eb-ab1f7836b5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:53.789075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8046a78-848d-42d2-a3eb-ab1f7836b5c1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:53.797942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8046a78-848d-42d2-a3eb-ab1f7836b5c1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:53.799437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8046a78-848d-42d2-a3eb-ab1f7836b5c1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-c8046a78-848d-42d2-a3eb-ab1f7836b5c1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:53.812736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:53.813805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:53.815361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:53.815361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:53.815361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:53.815361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:53.815361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:53.815361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:53.815361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:53.815361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:53.815361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:30:53.815361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:53.815361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 504/1023] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Sep 6 22:30:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:53.820480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e2166-4752-4864-95bf-bb38077abfef None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:53.823421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:53.915318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada69cd-90db-4984-bb4b-852ec0add6d3 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:53.915516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada69cd-90db-4984-bb4b-852ec0add6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:53.915763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aada69cd-90db-4984-bb4b-852ec0add6d3 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:53.917272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 505/1024] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:30:53.921191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88e2166-4752-4864-95bf-bb38077abfef None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:53.922077 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0be3e8-2e34-46de-8c10-8bc559781655 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:53.922356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0be3e8-2e34-46de-8c10-8bc559781655 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:53.922628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0be3e8-2e34-46de-8c10-8bc559781655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:53.942075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1f0be3e8-2e34-46de-8c10-8bc559781655 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:53.943290 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1f0be3e8-2e34-46de-8c10-8bc559781655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:53.961391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445a74a-8059-4c3b-9cec-19daa058b3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:53.961999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445a74a-8059-4c3b-9cec-19daa058b3cb None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:53.962274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445a74a-8059-4c3b-9cec-19daa058b3cb None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:53.962544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0445a74a-8059-4c3b-9cec-19daa058b3cb None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:53.963267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445a74a-8059-4c3b-9cec-19daa058b3cb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:53.963575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445a74a-8059-4c3b-9cec-19daa058b3cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:53.965027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0445a74a-8059-4c3b-9cec-19daa058b3cb None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:53.989852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 520/1025] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Sep 6 22:30:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:53.991950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445a74a-8059-4c3b-9cec-19daa058b3cb None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:54.018096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 506/1026] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:53 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:54.058408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:54.100639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0445a74a-8059-4c3b-9cec-19daa058b3cb None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:54.134641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a1a7e-67dd-4574-a178-b509fd0d60f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:54.135267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a1a7e-67dd-4574-a178-b509fd0d60f3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:54.135546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a1a7e-67dd-4574-a178-b509fd0d60f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:54.135819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415a1a7e-67dd-4574-a178-b509fd0d60f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:54.205224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a1a7e-67dd-4574-a178-b509fd0d60f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:54.206672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415a1a7e-67dd-4574-a178-b509fd0d60f3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:54.263270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:54.287659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0be3e8-2e34-46de-8c10-8bc559781655 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:54.314534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 521/1027] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:30:53 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 22:30:54.316220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a1a7e-67dd-4574-a178-b509fd0d60f3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:54.389458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0be3e8-2e34-46de-8c10-8bc559781655 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:54.406005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:54.419273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415a1a7e-67dd-4574-a178-b509fd0d60f3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:54.424264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bf885-bc44-4663-9c67-f93bd4d13e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:54.425440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bf885-bc44-4663-9c67-f93bd4d13e05 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:54.425715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bf885-bc44-4663-9c67-f93bd4d13e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:54.426123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8bf885-bc44-4663-9c67-f93bd4d13e05 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:54.427217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bf885-bc44-4663-9c67-f93bd4d13e05 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:54.428637 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8bf885-bc44-4663-9c67-f93bd4d13e05 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:54.434951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 507/1028] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:54.435699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bf885-bc44-4663-9c67-f93bd4d13e05 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:54.456572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b0165-aa28-4607-881a-9df03fab3f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:54.457176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b0165-aa28-4607-881a-9df03fab3f34 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:54.457427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b0165-aa28-4607-881a-9df03fab3f34 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:54.457694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225b0165-aa28-4607-881a-9df03fab3f34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:54.467846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b0165-aa28-4607-881a-9df03fab3f34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:54.469316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225b0165-aa28-4607-881a-9df03fab3f34 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-225b0165-aa28-4607-881a-9df03fab3f34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:54.484928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:54.485925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:54.486827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:54.486827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:54.486827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:54.486827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 522/1029] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:54.491492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b0165-aa28-4607-881a-9df03fab3f34 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:54.506910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:54.544954 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8bf885-bc44-4663-9c67-f93bd4d13e05 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:54.583576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cc948-118b-4c7e-9580-31e0ea132b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:54.584187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cc948-118b-4c7e-9580-31e0ea132b63 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:54.584464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cc948-118b-4c7e-9580-31e0ea132b63 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:54.584734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519cc948-118b-4c7e-9580-31e0ea132b63 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:54.585402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cc948-118b-4c7e-9580-31e0ea132b63 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:54.586818 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519cc948-118b-4c7e-9580-31e0ea132b63 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:54.602389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225b0165-aa28-4607-881a-9df03fab3f34 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:54.619403 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 508/1030] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:30:54 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:30:54.653729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7125dc-96ef-4094-9c22-d8b7690a4312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:54.654350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7125dc-96ef-4094-9c22-d8b7690a4312 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:54.654628 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7125dc-96ef-4094-9c22-d8b7690a4312 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:54.654902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7125dc-96ef-4094-9c22-d8b7690a4312 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:54.655555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7125dc-96ef-4094-9c22-d8b7690a4312 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:54.655860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7125dc-96ef-4094-9c22-d8b7690a4312 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:54.657301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7125dc-96ef-4094-9c22-d8b7690a4312 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:54.670955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 523/1031] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:54.677441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cc948-118b-4c7e-9580-31e0ea132b63 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:54.715795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:54.788248 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519cc948-118b-4c7e-9580-31e0ea132b63 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:54.826814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6243-dac8-4789-af99-1f4efd07c608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:54.827450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6243-dac8-4789-af99-1f4efd07c608 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:54.827762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6243-dac8-4789-af99-1f4efd07c608 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:54.828031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6243-dac8-4789-af99-1f4efd07c608 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:54.835444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd6243-dac8-4789-af99-1f4efd07c608 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:54.836897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd6243-dac8-4789-af99-1f4efd07c608 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e1bd6243-dac8-4789-af99-1f4efd07c608 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:54.848636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:54.849602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:54.850801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:54.850801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:54.850801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:54.850801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:54.850801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:54.850801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:54.850801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:54.850801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:54.850801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 509/1032] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:54.855893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7125dc-96ef-4094-9c22-d8b7690a4312 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:54.927503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:54.959641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7125dc-96ef-4094-9c22-d8b7690a4312 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:54.993111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e06a3b-0896-4972-98d1-4209c6793395 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:54.993626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e06a3b-0896-4972-98d1-4209c6793395 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:54.993898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e06a3b-0896-4972-98d1-4209c6793395 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:54.994168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e06a3b-0896-4972-98d1-4209c6793395 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:54.994839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e06a3b-0896-4972-98d1-4209c6793395 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:54.995177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e06a3b-0896-4972-98d1-4209c6793395 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:54.996540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e06a3b-0896-4972-98d1-4209c6793395 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:55.014273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 524/1033] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:55.015834 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6243-dac8-4789-af99-1f4efd07c608 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:55.113328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:55.120047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6243-dac8-4789-af99-1f4efd07c608 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:55.155524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca2c4ac-b97f-4c35-8ed7-2d276d5a9ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:55.156121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2c4ac-b97f-4c35-8ed7-2d276d5a9ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:55.158006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2c4ac-b97f-4c35-8ed7-2d276d5a9ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:55.158006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca2c4ac-b97f-4c35-8ed7-2d276d5a9ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:55.158006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2c4ac-b97f-4c35-8ed7-2d276d5a9ed7 None admin] RBAC: Adding query filter params (role.id=a2aaf3bf584d4607946a010016852567, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=b16947313dad4d59a97cc8b016ee54c1) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:55.158006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2c4ac-b97f-4c35-8ed7-2d276d5a9ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:55.159051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca2c4ac-b97f-4c35-8ed7-2d276d5a9ed7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:55.165472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 510/1034] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:30:55 2022] GET /identity/v3/role_assignments?user.id=b16947313dad4d59a97cc8b016ee54c1&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=a2aaf3bf584d4607946a010016852567 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:55.479672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8cd4d-e154-44e9-b94b-b8996852a8b2 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:55.480037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8cd4d-e154-44e9-b94b-b8996852a8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:55.480215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb8cd4d-e154-44e9-b94b-b8996852a8b2 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:55.481731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 525/1035] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:55.486708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc7ab-4754-40c9-b64f-643f83cb8c61 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:55.486996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc7ab-4754-40c9-b64f-643f83cb8c61 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:55.487327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35dc7ab-4754-40c9-b64f-643f83cb8c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:55.506027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d35dc7ab-4754-40c9-b64f-643f83cb8c61 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:55.507212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d35dc7ab-4754-40c9-b64f-643f83cb8c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:55.586543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 511/1036] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:55 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:55.850528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8cd4d-e154-44e9-b94b-b8996852a8b2 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:55.959801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb8cd4d-e154-44e9-b94b-b8996852a8b2 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:55.990628 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:55.996956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4ba63-c152-4788-aa79-2bbc696003b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:55.997586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ba63-c152-4788-aa79-2bbc696003b7 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:55.997849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ba63-c152-4788-aa79-2bbc696003b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:55.998149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4ba63-c152-4788-aa79-2bbc696003b7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:55.998732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4ba63-c152-4788-aa79-2bbc696003b7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:56.000202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4ba63-c152-4788-aa79-2bbc696003b7 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-aad4ba63-c152-4788-aa79-2bbc696003b7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:56.010629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:56.011640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:56.017484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 526/1037] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Sep 6 22:30:55 2022] GET /identity/v3/services/image => generated 86 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:56.020617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dc7ab-4754-40c9-b64f-643f83cb8c61 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:56.112532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:56.123436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35dc7ab-4754-40c9-b64f-643f83cb8c61 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:56.157590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca45fbb-05e7-47a6-837d-a566ca9bb5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:56.158193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca45fbb-05e7-47a6-837d-a566ca9bb5d7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:56.158473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca45fbb-05e7-47a6-837d-a566ca9bb5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:56.158741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca45fbb-05e7-47a6-837d-a566ca9bb5d7 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:56.159906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca45fbb-05e7-47a6-837d-a566ca9bb5d7 None admin] RBAC: Adding query filter params (name=image) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:56.160201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca45fbb-05e7-47a6-837d-a566ca9bb5d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:56.161751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ca45fbb-05e7-47a6-837d-a566ca9bb5d7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:56.166829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 512/1038] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:56.172342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4ba63-c152-4788-aa79-2bbc696003b7 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:56.279397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4ba63-c152-4788-aa79-2bbc696003b7 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:56.312815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a68221-b045-4619-a0ca-73b045d52c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:56.313421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a68221-b045-4619-a0ca-73b045d52c47 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:56.313697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a68221-b045-4619-a0ca-73b045d52c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:56.313966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a68221-b045-4619-a0ca-73b045d52c47 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:56.314588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a68221-b045-4619-a0ca-73b045d52c47 None admin] RBAC: Adding query filter params (type=image) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:56.314890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a68221-b045-4619-a0ca-73b045d52c47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:56.316843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a68221-b045-4619-a0ca-73b045d52c47 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:56.326441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 527/1039] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:56.327947 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca45fbb-05e7-47a6-837d-a566ca9bb5d7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:56.429613 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca45fbb-05e7-47a6-837d-a566ca9bb5d7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:56.463652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432fa3c9-a33a-4499-986b-5a1d8a9b06e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:56.464267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fa3c9-a33a-4499-986b-5a1d8a9b06e5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:56.464547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fa3c9-a33a-4499-986b-5a1d8a9b06e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:56.464846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432fa3c9-a33a-4499-986b-5a1d8a9b06e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:56.465544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fa3c9-a33a-4499-986b-5a1d8a9b06e5 None admin] RBAC: Adding query filter params (interface=public, service_id=e530db752b5245e487f7a7ae3080865e, region_id=RegionOne) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:56.465849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fa3c9-a33a-4499-986b-5a1d8a9b06e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:56.467212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432fa3c9-a33a-4499-986b-5a1d8a9b06e5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:56.477259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 513/1040] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Sep 6 22:30:56 2022] GET /identity/v3/endpoints?service_id=e530db752b5245e487f7a7ae3080865e&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:56.478864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a68221-b045-4619-a0ca-73b045d52c47 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:56.540584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:56.591171 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a68221-b045-4619-a0ca-73b045d52c47 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:56.625028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f080310-e83a-403e-8377-41223e5ed79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:56.625615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f080310-e83a-403e-8377-41223e5ed79a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:56.625886 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f080310-e83a-403e-8377-41223e5ed79a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:56.626151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f080310-e83a-403e-8377-41223e5ed79a None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:56.626737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f080310-e83a-403e-8377-41223e5ed79a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:56.628814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f080310-e83a-403e-8377-41223e5ed79a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:56.635853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 528/1041] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Sep 6 22:30:56 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:30:56.902296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb0628-22a3-4eab-ad05-3405c4236fcb None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:56.902713 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb0628-22a3-4eab-ad05-3405c4236fcb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:56.902828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fb0628-22a3-4eab-ad05-3405c4236fcb None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:56.904328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 514/1042] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:30:56.909835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d56c0f-9d55-41c6-92b4-a23eff323fa6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:56.910301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d56c0f-9d55-41c6-92b4-a23eff323fa6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:56.910301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d56c0f-9d55-41c6-92b4-a23eff323fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:56.929152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-b8d56c0f-9d55-41c6-92b4-a23eff323fa6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:56.930332 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b8d56c0f-9d55-41c6-92b4-a23eff323fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:57.002320 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 529/1043] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:56 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:30:57.280789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb0628-22a3-4eab-ad05-3405c4236fcb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:57.302042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:57.308343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e02a49-af61-487d-88b9-33029e3294d7 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:57.308631 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e02a49-af61-487d-88b9-33029e3294d7 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:57.308935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e02a49-af61-487d-88b9-33029e3294d7 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:57.310255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 530/1044] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:30:57.315251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b41637-5a44-4748-9c0e-81bf544a35e2 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:57.316219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b41637-5a44-4748-9c0e-81bf544a35e2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:57.316507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b41637-5a44-4748-9c0e-81bf544a35e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:57.334368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-76b41637-5a44-4748-9c0e-81bf544a35e2 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:57.335546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-76b41637-5a44-4748-9c0e-81bf544a35e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:57.387522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:57.397987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fb0628-22a3-4eab-ad05-3405c4236fcb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:57.408633 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 531/1045] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:57 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:57.433127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a3a33-9ebf-41c1-ba08-926dcc2b56dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:57.433755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a3a33-9ebf-41c1-ba08-926dcc2b56dd None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:57.434030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a3a33-9ebf-41c1-ba08-926dcc2b56dd None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:57.434298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a3a33-9ebf-41c1-ba08-926dcc2b56dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:57.435422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a3a33-9ebf-41c1-ba08-926dcc2b56dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:57.436907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a3a33-9ebf-41c1-ba08-926dcc2b56dd None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:57.438654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 515/1046] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:57.444698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b41637-5a44-4748-9c0e-81bf544a35e2 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:57.548007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b41637-5a44-4748-9c0e-81bf544a35e2 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:57.589689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d2b7f-5d6a-4493-95e9-39058e7413c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:57.590294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d2b7f-5d6a-4493-95e9-39058e7413c2 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:57.590570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d2b7f-5d6a-4493-95e9-39058e7413c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:57.590839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931d2b7f-5d6a-4493-95e9-39058e7413c2 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:57.591484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d2b7f-5d6a-4493-95e9-39058e7413c2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:57.592925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931d2b7f-5d6a-4493-95e9-39058e7413c2 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:57.623273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 532/1047] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:30:57 2022] POST /identity/v3/users => generated 285 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:30:57.681009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a3a33-9ebf-41c1-ba08-926dcc2b56dd None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:57.774669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:57.791436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a3a33-9ebf-41c1-ba08-926dcc2b56dd None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:57.828562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595e70c-80a7-4259-a030-7dec4a400d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:57.829141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e70c-80a7-4259-a030-7dec4a400d57 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:57.829470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e70c-80a7-4259-a030-7dec4a400d57 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:57.829756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595e70c-80a7-4259-a030-7dec4a400d57 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:57.841148 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595e70c-80a7-4259-a030-7dec4a400d57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-9595e70c-80a7-4259-a030-7dec4a400d57 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:30:57.846244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:57.847773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:30:57.848681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:57.851253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 516/1048] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Sep 6 22:30:57 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:57.857130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d2b7f-5d6a-4493-95e9-39058e7413c2 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:57.900721 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:57.963686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931d2b7f-5d6a-4493-95e9-39058e7413c2 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:57.997014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765737c-a9a0-4bab-b38c-9a7ecbbeebf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:57.997617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765737c-a9a0-4bab-b38c-9a7ecbbeebf0 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:57.997894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765737c-a9a0-4bab-b38c-9a7ecbbeebf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:57.998162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3765737c-a9a0-4bab-b38c-9a7ecbbeebf0 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:57.998821 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765737c-a9a0-4bab-b38c-9a7ecbbeebf0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:57.999122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765737c-a9a0-4bab-b38c-9a7ecbbeebf0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:58.000478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3765737c-a9a0-4bab-b38c-9a7ecbbeebf0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:58.009518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 533/1049] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Sep 6 22:30:57 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:58.015367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595e70c-80a7-4259-a030-7dec4a400d57 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:58.117577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595e70c-80a7-4259-a030-7dec4a400d57 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:58.151328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72790032-3d33-4a94-87f0-2b84d86d89ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:58.151964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72790032-3d33-4a94-87f0-2b84d86d89ab None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:58.152241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72790032-3d33-4a94-87f0-2b84d86d89ab None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:58.152510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72790032-3d33-4a94-87f0-2b84d86d89ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:58.163988 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72790032-3d33-4a94-87f0-2b84d86d89ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:58.165457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72790032-3d33-4a94-87f0-2b84d86d89ab None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:58.171786 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-72790032-3d33-4a94-87f0-2b84d86d89ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:58.181409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:58.182371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:58.183306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:58.183306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:58.183306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:58.183306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 517/1050] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:58.189308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765737c-a9a0-4bab-b38c-9a7ecbbeebf0 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:58.292565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3765737c-a9a0-4bab-b38c-9a7ecbbeebf0 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:58.294119 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:58.328007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d38df-2077-499d-8065-3753448bf0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:58.328605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d38df-2077-499d-8065-3753448bf0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:58.328907 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d38df-2077-499d-8065-3753448bf0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:58.329171 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2d38df-2077-499d-8065-3753448bf0d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:58.329794 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d38df-2077-499d-8065-3753448bf0d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:58.330087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d38df-2077-499d-8065-3753448bf0d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:58.332110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2d38df-2077-499d-8065-3753448bf0d8 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:58.343609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 534/1051] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:58.349706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72790032-3d33-4a94-87f0-2b84d86d89ab None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:58.451578 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72790032-3d33-4a94-87f0-2b84d86d89ab None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:58.485131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715b169-9452-4822-af63-942b632d1ac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:58.485754 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b169-9452-4822-af63-942b632d1ac5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:58.486026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b169-9452-4822-af63-942b632d1ac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:58.486292 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f715b169-9452-4822-af63-942b632d1ac5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:58.494090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715b169-9452-4822-af63-942b632d1ac5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:58.495509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f715b169-9452-4822-af63-942b632d1ac5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f715b169-9452-4822-af63-942b632d1ac5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:30:58.507318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:58.508289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:58.509608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:58.509608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:58.509608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:30:58.509608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:30:58.509608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:30:58.509608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:30:58.509608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:30:58.509608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:30:58.509608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:30:58.509608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:58.512841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 518/1052] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Sep 6 22:30:58 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:58.514369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d38df-2077-499d-8065-3753448bf0d8 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:58.625774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2d38df-2077-499d-8065-3753448bf0d8 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:58.660888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55e0314-8d11-494b-9780-f2155180e146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:58.661480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0314-8d11-494b-9780-f2155180e146 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:58.661750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0314-8d11-494b-9780-f2155180e146 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:58.662014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55e0314-8d11-494b-9780-f2155180e146 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:58.662721 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e0314-8d11-494b-9780-f2155180e146 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:58.663019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e0314-8d11-494b-9780-f2155180e146 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:58.664447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55e0314-8d11-494b-9780-f2155180e146 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:58.682198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 535/1053] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Sep 6 22:30:58 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:58.687671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715b169-9452-4822-af63-942b632d1ac5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:58.751761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:58.795322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f715b169-9452-4822-af63-942b632d1ac5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:58.829095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec14174-827e-4b73-95e6-71e618a9dc35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:58.829723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec14174-827e-4b73-95e6-71e618a9dc35 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:58.829990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec14174-827e-4b73-95e6-71e618a9dc35 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:58.830255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec14174-827e-4b73-95e6-71e618a9dc35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:58.830473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073de06-572d-4792-95bb-d0dd281e2ce3 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:58.830762 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073de06-572d-4792-95bb-d0dd281e2ce3 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:58.831806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073de06-572d-4792-95bb-d0dd281e2ce3 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:58.833405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 536/1054] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:30:58.838241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f92b5-fd5d-4ac8-862e-ae18f58550a6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:58.838462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f92b5-fd5d-4ac8-862e-ae18f58550a6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:58.838734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31f92b5-fd5d-4ac8-862e-ae18f58550a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:58.842481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec14174-827e-4b73-95e6-71e618a9dc35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:58.843910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec14174-827e-4b73-95e6-71e618a9dc35 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:58.857752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c31f92b5-fd5d-4ac8-862e-ae18f58550a6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:58.858925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c31f92b5-fd5d-4ac8-862e-ae18f58550a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-fec14174-827e-4b73-95e6-71e618a9dc35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:30:58.860062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:58.861122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:30:58.862033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:30:58.862033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:30:58.862033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:30:58.862033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 519/1055] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:58.868361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec14174-827e-4b73-95e6-71e618a9dc35 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:58.875786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:58.940822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 537/1056] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:58 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:30:58.973655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec14174-827e-4b73-95e6-71e618a9dc35 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:58.984497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:59.010061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ac4c6-f06c-4c77-b9e9-aade16d7e38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:59.010588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ac4c6-f06c-4c77-b9e9-aade16d7e38e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:59.010912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ac4c6-f06c-4c77-b9e9-aade16d7e38e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:59.011162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210ac4c6-f06c-4c77-b9e9-aade16d7e38e None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:59.011846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ac4c6-f06c-4c77-b9e9-aade16d7e38e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:59.012145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ac4c6-f06c-4c77-b9e9-aade16d7e38e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:59.013515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210ac4c6-f06c-4c77-b9e9-aade16d7e38e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:59.030393 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 520/1057] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:30:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:59.031666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f92b5-fd5d-4ac8-862e-ae18f58550a6 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:59.131265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31f92b5-fd5d-4ac8-862e-ae18f58550a6 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:59.165712 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc33773-5f1a-4bb6-9750-9e99b0462277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:59.166337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc33773-5f1a-4bb6-9750-9e99b0462277 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:59.166614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc33773-5f1a-4bb6-9750-9e99b0462277 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:59.166887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc33773-5f1a-4bb6-9750-9e99b0462277 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:59.173500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc33773-5f1a-4bb6-9750-9e99b0462277 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:59.175098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc33773-5f1a-4bb6-9750-9e99b0462277 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-9cc33773-5f1a-4bb6-9750-9e99b0462277 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:30:59.186047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:59.186952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:59.187868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:30:59.187868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:30:59.187868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:30:59.187868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:30:59.187868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:30:59.187868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:30:59.187868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:30:59.187868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:59.192564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 538/1058] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:30:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:30:59.193800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ac4c6-f06c-4c77-b9e9-aade16d7e38e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:59.211167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc33773-5f1a-4bb6-9750-9e99b0462277 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:59.227123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:59.302827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210ac4c6-f06c-4c77-b9e9-aade16d7e38e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:59.339167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc33773-5f1a-4bb6-9750-9e99b0462277 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:59.339672 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50645073-7daf-4ac6-b8a8-3885e557025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:59.340283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50645073-7daf-4ac6-b8a8-3885e557025f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:59.340561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50645073-7daf-4ac6-b8a8-3885e557025f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:59.340862 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50645073-7daf-4ac6-b8a8-3885e557025f None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:59.341647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50645073-7daf-4ac6-b8a8-3885e557025f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:59.341956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50645073-7daf-4ac6-b8a8-3885e557025f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:59.343379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50645073-7daf-4ac6-b8a8-3885e557025f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:59.355097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 521/1059] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:30:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:30:59.361401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50645073-7daf-4ac6-b8a8-3885e557025f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:59.380105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37d9f-c043-4254-ab65-de58987924e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:59.380742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37d9f-c043-4254-ab65-de58987924e8 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:59.381051 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37d9f-c043-4254-ab65-de58987924e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:59.381320 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37d9f-c043-4254-ab65-de58987924e8 None admin] PATH_INFO: `/v3/services/image` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:59.381909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d37d9f-c043-4254-ab65-de58987924e8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:59.383290 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d37d9f-c043-4254-ab65-de58987924e8 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-65d37d9f-c043-4254-ab65-de58987924e8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:30:59.392513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:30:59.394335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:30:59.395250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 539/1060] 10.222.0.39 () {56 vars in 1105 bytes} [Tue Sep 6 22:30:59 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:30:59.399042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:59.400637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37d9f-c043-4254-ab65-de58987924e8 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:59.476362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50645073-7daf-4ac6-b8a8-3885e557025f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:59.481403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:59.512655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37d9f-c043-4254-ab65-de58987924e8 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:59.513826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2449a9-b21c-461d-b6a1-d0add22033f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:59.514605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2449a9-b21c-461d-b6a1-d0add22033f7 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:59.515006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2449a9-b21c-461d-b6a1-d0add22033f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:59.515278 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2449a9-b21c-461d-b6a1-d0add22033f7 None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/b16947313dad4d59a97cc8b016ee54c1/roles/a2aaf3bf584d4607946a010016852567` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:59.517553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2449a9-b21c-461d-b6a1-d0add22033f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=b16947313dad4d59a97cc8b016ee54c1, role_id=a2aaf3bf584d4607946a010016852567)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:59.518997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2449a9-b21c-461d-b6a1-d0add22033f7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:59.535956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 522/1061] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:30:59 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/b16947313dad4d59a97cc8b016ee54c1/roles/a2aaf3bf584d4607946a010016852567 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:30:59.553838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfddf50-5a16-41f6-b294-8f5a32c0ca96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:59.554391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfddf50-5a16-41f6-b294-8f5a32c0ca96 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:59.554668 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfddf50-5a16-41f6-b294-8f5a32c0ca96 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:59.554935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfddf50-5a16-41f6-b294-8f5a32c0ca96 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:59.555584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfddf50-5a16-41f6-b294-8f5a32c0ca96 None admin] RBAC: Adding query filter params (name=image) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:59.555886 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfddf50-5a16-41f6-b294-8f5a32c0ca96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:59.557331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbfddf50-5a16-41f6-b294-8f5a32c0ca96 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:59.568217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 540/1062] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:59.569446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2449a9-b21c-461d-b6a1-d0add22033f7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:59.676517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:59.689991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2449a9-b21c-461d-b6a1-d0add22033f7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:59.730831 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0a492b-2d10-44f1-a975-fb73b5ccd358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:59.731467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a492b-2d10-44f1-a975-fb73b5ccd358 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:59.731750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a492b-2d10-44f1-a975-fb73b5ccd358 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:59.732023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0a492b-2d10-44f1-a975-fb73b5ccd358 None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:59.732652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a492b-2d10-44f1-a975-fb73b5ccd358 None admin] RBAC: Adding query filter params (type=image) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:30:59.733033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a492b-2d10-44f1-a975-fb73b5ccd358 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:59.734464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0a492b-2d10-44f1-a975-fb73b5ccd358 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:59.739712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 523/1063] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:30:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:30:59.745706 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfddf50-5a16-41f6-b294-8f5a32c0ca96 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:30:59.853391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfddf50-5a16-41f6-b294-8f5a32c0ca96 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:30:59.877097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19845e-b77e-407f-88db-46a81553e093 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:59.877382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19845e-b77e-407f-88db-46a81553e093 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:59.878102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19845e-b77e-407f-88db-46a81553e093 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:59.880533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 524/1064] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:30:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:30:59.886222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2727e0e-0cb5-443e-9f9c-2948aa5de9c5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:59.886575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2727e0e-0cb5-443e-9f9c-2948aa5de9c5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:59.886925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2727e0e-0cb5-443e-9f9c-2948aa5de9c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:59.890306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4a697-f67b-4134-8e41-4830b7840882 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:30:59.890908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4a697-f67b-4134-8e41-4830b7840882 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:30:59.891217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4a697-f67b-4134-8e41-4830b7840882 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:30:59.891464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4a697-f67b-4134-8e41-4830b7840882 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:30:59.892087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4a697-f67b-4134-8e41-4830b7840882 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:30:59.894553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4a697-f67b-4134-8e41-4830b7840882 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:30:59.905092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:30:59.906246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a2727e0e-0cb5-443e-9f9c-2948aa5de9c5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:30:59.907424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a2727e0e-0cb5-443e-9f9c-2948aa5de9c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:30:59.921514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 541/1065] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:30:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:31:00.000519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 525/1066] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:30:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:31:00.269629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4a697-f67b-4134-8e41-4830b7840882 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:00.390670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4a697-f67b-4134-8e41-4830b7840882 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:00.429231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6c1ac-8cb5-45ec-9398-19ebb7e14d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:00.429869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6c1ac-8cb5-45ec-9398-19ebb7e14d2f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:00.430144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6c1ac-8cb5-45ec-9398-19ebb7e14d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:00.430416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d6c1ac-8cb5-45ec-9398-19ebb7e14d2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:00.443055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4d6c1ac-8cb5-45ec-9398-19ebb7e14d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-b4d6c1ac-8cb5-45ec-9398-19ebb7e14d2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:31:00.448064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:00.449044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:00.449893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:00.453085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 542/1067] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:31:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:00.455193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2727e0e-0cb5-443e-9f9c-2948aa5de9c5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:00.536873 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:00.560093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2727e0e-0cb5-443e-9f9c-2948aa5de9c5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:00.599255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0da0d-523e-4815-9241-a873662a0b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:00.599864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0da0d-523e-4815-9241-a873662a0b87 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:00.600144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0da0d-523e-4815-9241-a873662a0b87 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:00.600415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0da0d-523e-4815-9241-a873662a0b87 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:00.601106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0da0d-523e-4815-9241-a873662a0b87 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:00.601409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0da0d-523e-4815-9241-a873662a0b87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:00.602777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0da0d-523e-4815-9241-a873662a0b87 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:00.616235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 526/1068] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:31:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:00.617391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6c1ac-8cb5-45ec-9398-19ebb7e14d2f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:00.675621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:00.728843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d6c1ac-8cb5-45ec-9398-19ebb7e14d2f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:00.762632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91283a0-c72d-46ea-8c41-199b669c5509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:00.763263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91283a0-c72d-46ea-8c41-199b669c5509 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:00.763614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91283a0-c72d-46ea-8c41-199b669c5509 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:00.763894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91283a0-c72d-46ea-8c41-199b669c5509 None admin] PATH_INFO: `/v3/users/99c61cd7ff3a41e49998918f554ce9fa` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:00.777441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91283a0-c72d-46ea-8c41-199b669c5509 None admin] RBAC: Authorizing `identity:get_user(user_id=99c61cd7ff3a41e49998918f554ce9fa)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:00.778869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91283a0-c72d-46ea-8c41-199b669c5509 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:00.780659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 543/1069] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:00 2022] GET /identity/v3/users/99c61cd7ff3a41e49998918f554ce9fa => generated 285 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:00.787014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0da0d-523e-4815-9241-a873662a0b87 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:00.889941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0da0d-523e-4815-9241-a873662a0b87 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:00.923103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06517d33-1c4b-45a7-a4e2-58747d0c61d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:00.923708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06517d33-1c4b-45a7-a4e2-58747d0c61d1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:00.923986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06517d33-1c4b-45a7-a4e2-58747d0c61d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:00.924254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06517d33-1c4b-45a7-a4e2-58747d0c61d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:00.936064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:00.995206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06517d33-1c4b-45a7-a4e2-58747d0c61d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:00.996858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06517d33-1c4b-45a7-a4e2-58747d0c61d1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:01.096057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 527/1070] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:31:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:31:01.101729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91283a0-c72d-46ea-8c41-199b669c5509 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:01.205746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91283a0-c72d-46ea-8c41-199b669c5509 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:01.219767 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:01.242203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f937f4-af47-40e8-af87-757ad7ca780f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:01.242832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f937f4-af47-40e8-af87-757ad7ca780f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:01.243107 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f937f4-af47-40e8-af87-757ad7ca780f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:01.243374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f937f4-af47-40e8-af87-757ad7ca780f None admin] PATH_INFO: `/v3/projects/c0e94ffe9f09419caf394da7273faf9e` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:01.244466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f937f4-af47-40e8-af87-757ad7ca780f None admin] RBAC: Authorizing `identity:get_project(project_id=c0e94ffe9f09419caf394da7273faf9e)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:01.245902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f937f4-af47-40e8-af87-757ad7ca780f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:01.247672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 544/1071] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:01 2022] GET /identity/v3/projects/c0e94ffe9f09419caf394da7273faf9e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:01.253993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06517d33-1c4b-45a7-a4e2-58747d0c61d1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:01.298140 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:01.356230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06517d33-1c4b-45a7-a4e2-58747d0c61d1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:01.389860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b5427f-4422-4642-94a2-b8c172f4c5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:01.390456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5427f-4422-4642-94a2-b8c172f4c5b8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:01.390733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5427f-4422-4642-94a2-b8c172f4c5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:01.390997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b5427f-4422-4642-94a2-b8c172f4c5b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:01.391700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5427f-4422-4642-94a2-b8c172f4c5b8 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=c0e94ffe9f09419caf394da7273faf9e, user.id=99c61cd7ff3a41e49998918f554ce9fa) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:01.391997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5427f-4422-4642-94a2-b8c172f4c5b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:01.393956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b5427f-4422-4642-94a2-b8c172f4c5b8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:01.399331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 528/1072] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:31:01 2022] GET /identity/v3/role_assignments?user.id=99c61cd7ff3a41e49998918f554ce9fa&scope.project.id=c0e94ffe9f09419caf394da7273faf9e&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:01.740207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b15e6-c366-48a5-9010-9494cb04eb85 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:01.740700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b15e6-c366-48a5-9010-9494cb04eb85 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:01.740700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5b15e6-c366-48a5-9010-9494cb04eb85 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:01.742076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 545/1073] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:01.747449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7c23a-c537-4ed4-b8c2-02bce0cd3fbc None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:01.747918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7c23a-c537-4ed4-b8c2-02bce0cd3fbc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:01.747996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f7c23a-c537-4ed4-b8c2-02bce0cd3fbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:01.770101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-02f7c23a-c537-4ed4-b8c2-02bce0cd3fbc None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:01.771284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-02f7c23a-c537-4ed4-b8c2-02bce0cd3fbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:01.839617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 529/1074] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:31:02.107791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b15e6-c366-48a5-9010-9494cb04eb85 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:02.129118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e080d-99c0-47c0-90e5-03f12ec65f82 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:02.129406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e080d-99c0-47c0-90e5-03f12ec65f82 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:02.129679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674e080d-99c0-47c0-90e5-03f12ec65f82 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:02.131307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 530/1075] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:31:02.136406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1639a8-8fa7-469e-825d-ee44da8ab122 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:02.136688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1639a8-8fa7-469e-825d-ee44da8ab122 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:02.137009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1639a8-8fa7-469e-825d-ee44da8ab122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:02.155421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3a1639a8-8fa7-469e-825d-ee44da8ab122 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:02.156603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3a1639a8-8fa7-469e-825d-ee44da8ab122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:02.188543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:02.224170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5b15e6-c366-48a5-9010-9494cb04eb85 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:02.231671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 531/1076] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:31:02.260506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac83f7-1306-468d-996a-bd4666336695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:02.261190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac83f7-1306-468d-996a-bd4666336695 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:02.261470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac83f7-1306-468d-996a-bd4666336695 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:02.261743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daac83f7-1306-468d-996a-bd4666336695 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:02.274912 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daac83f7-1306-468d-996a-bd4666336695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-daac83f7-1306-468d-996a-bd4666336695 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:31:02.279841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:02.280799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:02.285637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 546/1077] 10.222.0.39 () {56 vars in 1120 bytes} [Tue Sep 6 22:31:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:02.287785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1639a8-8fa7-469e-825d-ee44da8ab122 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:02.388587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1639a8-8fa7-469e-825d-ee44da8ab122 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:02.422119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0efce0-aaf1-4ed3-9e26-7d1909122130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:02.422723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0efce0-aaf1-4ed3-9e26-7d1909122130 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:02.423016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0efce0-aaf1-4ed3-9e26-7d1909122130 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:02.423275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0efce0-aaf1-4ed3-9e26-7d1909122130 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:02.423933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0efce0-aaf1-4ed3-9e26-7d1909122130 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:02.424235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0efce0-aaf1-4ed3-9e26-7d1909122130 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:02.425925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0efce0-aaf1-4ed3-9e26-7d1909122130 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:02.438233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 532/1078] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Sep 6 22:31:02 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:02.439825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac83f7-1306-468d-996a-bd4666336695 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:02.483652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:02.499349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0efce0-aaf1-4ed3-9e26-7d1909122130 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:02.530289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:02.547263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daac83f7-1306-468d-996a-bd4666336695 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:02.583974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae8b2f-8dd2-47dc-877d-8b068ac8cbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:02.584600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae8b2f-8dd2-47dc-877d-8b068ac8cbf3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:02.584920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae8b2f-8dd2-47dc-877d-8b068ac8cbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:02.585190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae8b2f-8dd2-47dc-877d-8b068ac8cbf3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:02.596976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae8b2f-8dd2-47dc-877d-8b068ac8cbf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:02.598726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae8b2f-8dd2-47dc-877d-8b068ac8cbf3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-b0ae8b2f-8dd2-47dc-877d-8b068ac8cbf3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:31:02.615665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:02.616657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:31:02.622334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:31:02.622334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:31:02.622334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:02.622334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 547/1079] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:31:02.622644 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0efce0-aaf1-4ed3-9e26-7d1909122130 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:02.623868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae8b2f-8dd2-47dc-877d-8b068ac8cbf3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:02.656838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414cdfc-5d95-4e29-9de3-c6f77e342f0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:02.657339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414cdfc-5d95-4e29-9de3-c6f77e342f0c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:02.657617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414cdfc-5d95-4e29-9de3-c6f77e342f0c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:02.657889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414cdfc-5d95-4e29-9de3-c6f77e342f0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:02.719091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:02.728851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae8b2f-8dd2-47dc-877d-8b068ac8cbf3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:02.728987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414cdfc-5d95-4e29-9de3-c6f77e342f0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:02.730563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414cdfc-5d95-4e29-9de3-c6f77e342f0c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:02.763470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadeb581-2687-45cb-92e0-db1e0f95b0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:02.764076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadeb581-2687-45cb-92e0-db1e0f95b0b0 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:02.764353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadeb581-2687-45cb-92e0-db1e0f95b0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:02.764460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:02.764852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadeb581-2687-45cb-92e0-db1e0f95b0b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:02.765273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadeb581-2687-45cb-92e0-db1e0f95b0b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:02.765660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadeb581-2687-45cb-92e0-db1e0f95b0b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:02.767315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadeb581-2687-45cb-92e0-db1e0f95b0b0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:02.779404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 548/1080] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:31:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:31:02.785208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadeb581-2687-45cb-92e0-db1e0f95b0b0 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:02.847685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 533/1081] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:31:02 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 22:31:02.853763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414cdfc-5d95-4e29-9de3-c6f77e342f0c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:02.888684 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadeb581-2687-45cb-92e0-db1e0f95b0b0 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:02.926557 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df58f7-b1b4-4648-8b27-77048617e1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:02.927209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df58f7-b1b4-4648-8b27-77048617e1ba None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:02.927475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df58f7-b1b4-4648-8b27-77048617e1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:02.927744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01df58f7-b1b4-4648-8b27-77048617e1ba None admin] PATH_INFO: `/v3/users/nova` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:02.934771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df58f7-b1b4-4648-8b27-77048617e1ba None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:02.936185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01df58f7-b1b4-4648-8b27-77048617e1ba None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-01df58f7-b1b4-4648-8b27-77048617e1ba None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:31:02.948034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:02.949154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:02.950218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:02.950218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:02.950218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:31:02.950218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:31:02.950218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:31:02.950218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:31:02.950218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:31:02.950218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:31:02.950218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 06 22:31:02.950218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:02.950218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 549/1082] 10.222.0.39 () {56 vars in 1093 bytes} [Tue Sep 6 22:31:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:31:02.954982 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df58f7-b1b4-4648-8b27-77048617e1ba None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:02.965668 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414cdfc-5d95-4e29-9de3-c6f77e342f0c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:02.972255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:02.993370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:03.001190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f7374-e284-4b35-b05e-901757d8d759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:03.001821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f7374-e284-4b35-b05e-901757d8d759 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:03.002096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f7374-e284-4b35-b05e-901757d8d759 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:03.002364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396f7374-e284-4b35-b05e-901757d8d759 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:03.013433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f7374-e284-4b35-b05e-901757d8d759 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:03.014859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396f7374-e284-4b35-b05e-901757d8d759 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-396f7374-e284-4b35-b05e-901757d8d759 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:31:03.030273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:03.031244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:31:03.032158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:31:03.032158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:31:03.032158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:31:03.032158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 534/1083] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:31:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:31:03.037982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f7374-e284-4b35-b05e-901757d8d759 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:03.069485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01df58f7-b1b4-4648-8b27-77048617e1ba None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:03.103524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd2bd4-0b0e-4190-b7a2-7d78558b7abf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:03.104119 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd2bd4-0b0e-4190-b7a2-7d78558b7abf None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:03.104393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd2bd4-0b0e-4190-b7a2-7d78558b7abf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:03.104659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccd2bd4-0b0e-4190-b7a2-7d78558b7abf None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:03.105400 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd2bd4-0b0e-4190-b7a2-7d78558b7abf None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:03.105700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd2bd4-0b0e-4190-b7a2-7d78558b7abf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:03.107105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eccd2bd4-0b0e-4190-b7a2-7d78558b7abf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:03.125388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 550/1084] 10.222.0.39 () {56 vars in 1133 bytes} [Tue Sep 6 22:31:02 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:03.130913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd2bd4-0b0e-4190-b7a2-7d78558b7abf None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:03.147475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396f7374-e284-4b35-b05e-901757d8d759 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:03.184746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aaef86-9b15-42b1-887f-62a9287ba4f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:03.185298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaef86-9b15-42b1-887f-62a9287ba4f4 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:03.185577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaef86-9b15-42b1-887f-62a9287ba4f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:03.185848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aaef86-9b15-42b1-887f-62a9287ba4f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:03.186488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aaef86-9b15-42b1-887f-62a9287ba4f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:03.186785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aaef86-9b15-42b1-887f-62a9287ba4f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:03.188153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aaef86-9b15-42b1-887f-62a9287ba4f4 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:03.201437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 535/1085] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:31:03.213038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:03.249210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eccd2bd4-0b0e-4190-b7a2-7d78558b7abf None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:03.293171 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a1bd53-ec4f-4e17-a679-7c83b5855799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:03.293785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1bd53-ec4f-4e17-a679-7c83b5855799 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:03.294056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1bd53-ec4f-4e17-a679-7c83b5855799 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:03.294320 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a1bd53-ec4f-4e17-a679-7c83b5855799 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:03.367459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1bd53-ec4f-4e17-a679-7c83b5855799 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:03.368919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a1bd53-ec4f-4e17-a679-7c83b5855799 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:03.442028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:03.475797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 551/1086] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:31:03 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 06 22:31:03.482059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aaef86-9b15-42b1-887f-62a9287ba4f4 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:03.533616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:03.592200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aaef86-9b15-42b1-887f-62a9287ba4f4 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:03.619777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465344cd-b170-4b5e-b7e5-d4fbefb1e5b6 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:03.620057 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465344cd-b170-4b5e-b7e5-d4fbefb1e5b6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:03.620332 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465344cd-b170-4b5e-b7e5-d4fbefb1e5b6 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:03.621700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 552/1087] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:03.625898 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5edf07-fe97-4936-9674-1a9925613a38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:03.626525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5edf07-fe97-4936-9674-1a9925613a38 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:03.626682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e51e4c-d151-4bf1-8ae7-1d90a6767bea None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:03.626803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5edf07-fe97-4936-9674-1a9925613a38 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:03.626963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e51e4c-d151-4bf1-8ae7-1d90a6767bea None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:03.627061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5edf07-fe97-4936-9674-1a9925613a38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:03.627235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e51e4c-d151-4bf1-8ae7-1d90a6767bea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:03.638097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5edf07-fe97-4936-9674-1a9925613a38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:03.639521 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d5edf07-fe97-4936-9674-1a9925613a38 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:03.645711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-f4e51e4c-d151-4bf1-8ae7-1d90a6767bea None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:03.646891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f4e51e4c-d151-4bf1-8ae7-1d90a6767bea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-0d5edf07-fe97-4936-9674-1a9925613a38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:31:03.654021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:03.654986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:31:03.655980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:31:03.655980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:31:03.655980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:31:03.655980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 536/1088] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:31:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:03.660681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5edf07-fe97-4936-9674-1a9925613a38 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:03.715426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 553/1089] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:31:03.753320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:03.762441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5edf07-fe97-4936-9674-1a9925613a38 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:03.795075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf36fc-7d3e-4411-8c4a-374a2685ca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:03.795671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf36fc-7d3e-4411-8c4a-374a2685ca73 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:03.795942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf36fc-7d3e-4411-8c4a-374a2685ca73 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:03.796209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cf36fc-7d3e-4411-8c4a-374a2685ca73 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:03.796878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf36fc-7d3e-4411-8c4a-374a2685ca73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:03.797178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf36fc-7d3e-4411-8c4a-374a2685ca73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:03.798532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cf36fc-7d3e-4411-8c4a-374a2685ca73 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:03.809827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 537/1090] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:31:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:31:03.817182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e51e4c-d151-4bf1-8ae7-1d90a6767bea None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:03.911432 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:03.914369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e51e4c-d151-4bf1-8ae7-1d90a6767bea None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:03.948682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215c8d7-c053-4089-a1d2-6c1a5e1ad1f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:03.949333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c8d7-c053-4089-a1d2-6c1a5e1ad1f1 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:03.949612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c8d7-c053-4089-a1d2-6c1a5e1ad1f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:03.949882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a215c8d7-c053-4089-a1d2-6c1a5e1ad1f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:03.957032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215c8d7-c053-4089-a1d2-6c1a5e1ad1f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:03.958570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a215c8d7-c053-4089-a1d2-6c1a5e1ad1f1 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-a215c8d7-c053-4089-a1d2-6c1a5e1ad1f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:31:03.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:03.971424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:03.972498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:31:03.972498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:31:03.972498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:31:03.972498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:31:03.972498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:31:03.972498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:31:03.972498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:31:03.972498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:03.972498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 554/1091] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:31:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:03.977688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf36fc-7d3e-4411-8c4a-374a2685ca73 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:03.986244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215c8d7-c053-4089-a1d2-6c1a5e1ad1f1 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:04.078087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cf36fc-7d3e-4411-8c4a-374a2685ca73 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:04.083506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a215c8d7-c053-4089-a1d2-6c1a5e1ad1f1 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:04.111826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc4492-c639-43e3-81e6-9d4bf86ad60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:04.112979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc4492-c639-43e3-81e6-9d4bf86ad60d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:04.113255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc4492-c639-43e3-81e6-9d4bf86ad60d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:04.113524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fc4492-c639-43e3-81e6-9d4bf86ad60d None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:04.114210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc4492-c639-43e3-81e6-9d4bf86ad60d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:04.114512 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc4492-c639-43e3-81e6-9d4bf86ad60d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:04.115925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fc4492-c639-43e3-81e6-9d4bf86ad60d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:04.116656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64308e2f-c418-45bb-b2fb-eeeaf9bbe4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:04.117355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64308e2f-c418-45bb-b2fb-eeeaf9bbe4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:04.117636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64308e2f-c418-45bb-b2fb-eeeaf9bbe4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:04.117904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64308e2f-c418-45bb-b2fb-eeeaf9bbe4c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:04.127072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 538/1092] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:31:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:04.128038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64308e2f-c418-45bb-b2fb-eeeaf9bbe4c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-64308e2f-c418-45bb-b2fb-eeeaf9bbe4c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:31:04.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:04.134415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:04.135635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:04.136462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc4492-c639-43e3-81e6-9d4bf86ad60d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:04.136555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 555/1093] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:31:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:31:04.140600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64308e2f-c418-45bb-b2fb-eeeaf9bbe4c4 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:04.150296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:04.164279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:04.233920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fc4492-c639-43e3-81e6-9d4bf86ad60d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:04.241398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64308e2f-c418-45bb-b2fb-eeeaf9bbe4c4 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:04.267634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b671b-37ee-4a24-8b5d-3dca01fc1d71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:04.268233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b671b-37ee-4a24-8b5d-3dca01fc1d71 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:04.268554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b671b-37ee-4a24-8b5d-3dca01fc1d71 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:04.268929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267b671b-37ee-4a24-8b5d-3dca01fc1d71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:04.269711 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b671b-37ee-4a24-8b5d-3dca01fc1d71 None admin] RBAC: Adding query filter params (role.id=a2aaf3bf584d4607946a010016852567, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=b16947313dad4d59a97cc8b016ee54c1) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:04.269936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b671b-37ee-4a24-8b5d-3dca01fc1d71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:04.271348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267b671b-37ee-4a24-8b5d-3dca01fc1d71 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:04.275002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f9972-0367-48c2-97d2-285db57199f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:04.275586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f9972-0367-48c2-97d2-285db57199f7 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:04.275857 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f9972-0367-48c2-97d2-285db57199f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:04.276120 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5f9972-0367-48c2-97d2-285db57199f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:04.276779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f9972-0367-48c2-97d2-285db57199f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:04.277013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 539/1094] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:31:04 2022] GET /identity/v3/role_assignments?user.id=b16947313dad4d59a97cc8b016ee54c1&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=a2aaf3bf584d4607946a010016852567 => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:31:04.277111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f9972-0367-48c2-97d2-285db57199f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:04.278397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5f9972-0367-48c2-97d2-285db57199f7 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:04.287021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 556/1095] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:31:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:04.293049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b671b-37ee-4a24-8b5d-3dca01fc1d71 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:04.374988 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:04.399470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267b671b-37ee-4a24-8b5d-3dca01fc1d71 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:04.435014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdeb30-ab1c-4864-82af-49d9527a3412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:04.435651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdeb30-ab1c-4864-82af-49d9527a3412 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:04.435928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdeb30-ab1c-4864-82af-49d9527a3412 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:04.436205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffdeb30-ab1c-4864-82af-49d9527a3412 None admin] PATH_INFO: `/v3/users/99c61cd7ff3a41e49998918f554ce9fa` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:04.437364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdeb30-ab1c-4864-82af-49d9527a3412 None admin] RBAC: Authorizing `identity:get_user(user_id=99c61cd7ff3a41e49998918f554ce9fa)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:04.438825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffdeb30-ab1c-4864-82af-49d9527a3412 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:04.441222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 540/1096] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:04 2022] GET /identity/v3/users/99c61cd7ff3a41e49998918f554ce9fa => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:04.446885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f9972-0367-48c2-97d2-285db57199f7 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:04.532733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:04.551924 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5f9972-0367-48c2-97d2-285db57199f7 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:04.591592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7824570-f02c-466e-adc8-180bfcd7a1a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:04.592218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7824570-f02c-466e-adc8-180bfcd7a1a9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:04.592489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7824570-f02c-466e-adc8-180bfcd7a1a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:04.592835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7824570-f02c-466e-adc8-180bfcd7a1a9 None admin] PATH_INFO: `/v3/projects/c0e94ffe9f09419caf394da7273faf9e` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:04.593889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7824570-f02c-466e-adc8-180bfcd7a1a9 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e94ffe9f09419caf394da7273faf9e)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:04.595293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7824570-f02c-466e-adc8-180bfcd7a1a9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:04.597603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 557/1097] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:04 2022] GET /identity/v3/projects/c0e94ffe9f09419caf394da7273faf9e => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:04.603707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdeb30-ab1c-4864-82af-49d9527a3412 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:04.709342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffdeb30-ab1c-4864-82af-49d9527a3412 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:04.744170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1af5b-7e09-4309-97c4-7078a124c7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:04.744896 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1af5b-7e09-4309-97c4-7078a124c7fa None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:04.745174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1af5b-7e09-4309-97c4-7078a124c7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:04.745439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1af5b-7e09-4309-97c4-7078a124c7fa None admin] PATH_INFO: `/v3/projects/c0e94ffe9f09419caf394da7273faf9e/users/99c61cd7ff3a41e49998918f554ce9fa/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:04.747433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1af5b-7e09-4309-97c4-7078a124c7fa None admin] RBAC: Authorizing `identity:create_grant(project_id=c0e94ffe9f09419caf394da7273faf9e, user_id=99c61cd7ff3a41e49998918f554ce9fa, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:04.749004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1af5b-7e09-4309-97c4-7078a124c7fa None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:04.762777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 541/1098] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:31:04 2022] PUT /identity/v3/projects/c0e94ffe9f09419caf394da7273faf9e/users/99c61cd7ff3a41e49998918f554ce9fa/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:31:06.938701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f381-1558-43a2-a455-e2b9c3405597 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:06.939223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f381-1558-43a2-a455-e2b9c3405597 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:06.939304 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec4f381-1558-43a2-a455-e2b9c3405597 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:06.941743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 558/1099] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:06.948505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8808f-3671-424e-993c-bb036ca47e9c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:06.949181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8808f-3671-424e-993c-bb036ca47e9c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:06.949181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e8808f-3671-424e-993c-bb036ca47e9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:06.956946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:06.968373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-74e8808f-3671-424e-993c-bb036ca47e9c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:06.969586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-74e8808f-3671-424e-993c-bb036ca47e9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:07.035184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 542/1100] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:07.336586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4f381-1558-43a2-a455-e2b9c3405597 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:07.434934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec4f381-1558-43a2-a455-e2b9c3405597 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:07.469132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84812b2d-be8d-45d0-9066-762287adf694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:07.469747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84812b2d-be8d-45d0-9066-762287adf694 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:07.470018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84812b2d-be8d-45d0-9066-762287adf694 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:07.470283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84812b2d-be8d-45d0-9066-762287adf694 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:07.480463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84812b2d-be8d-45d0-9066-762287adf694 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-84812b2d-be8d-45d0-9066-762287adf694 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:31:07.485180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:07.486360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:07.488442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 559/1101] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:31:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:07.493272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8808f-3671-424e-993c-bb036ca47e9c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:07.589894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e8808f-3671-424e-993c-bb036ca47e9c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:07.623110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df4dd3-f652-4322-b7e1-2e71ddb1f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:07.623711 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df4dd3-f652-4322-b7e1-2e71ddb1f194 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:07.623989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df4dd3-f652-4322-b7e1-2e71ddb1f194 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:07.624258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54df4dd3-f652-4322-b7e1-2e71ddb1f194 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:07.624942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df4dd3-f652-4322-b7e1-2e71ddb1f194 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:07.625238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df4dd3-f652-4322-b7e1-2e71ddb1f194 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:07.626591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54df4dd3-f652-4322-b7e1-2e71ddb1f194 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:07.628384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:07.636094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 543/1102] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:31:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:07.642162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84812b2d-be8d-45d0-9066-762287adf694 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:07.655551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:07.740882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84812b2d-be8d-45d0-9066-762287adf694 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:07.780041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca99a3e-bfa0-4f2f-b0cf-61d0594368cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:07.780664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca99a3e-bfa0-4f2f-b0cf-61d0594368cb None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:07.780967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca99a3e-bfa0-4f2f-b0cf-61d0594368cb None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:07.781237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca99a3e-bfa0-4f2f-b0cf-61d0594368cb None admin] PATH_INFO: `/v3/users/99c61cd7ff3a41e49998918f554ce9fa` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:07.782359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca99a3e-bfa0-4f2f-b0cf-61d0594368cb None admin] RBAC: Authorizing `identity:get_user(user_id=99c61cd7ff3a41e49998918f554ce9fa)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:07.783777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca99a3e-bfa0-4f2f-b0cf-61d0594368cb None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:07.785429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 560/1103] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:07 2022] GET /identity/v3/users/99c61cd7ff3a41e49998918f554ce9fa => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:07.791369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df4dd3-f652-4322-b7e1-2e71ddb1f194 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:07.894096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54df4dd3-f652-4322-b7e1-2e71ddb1f194 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:07.926572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff9f5b-d856-422c-b182-e3aa30feeb27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:07.927166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9f5b-d856-422c-b182-e3aa30feeb27 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:07.927441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9f5b-d856-422c-b182-e3aa30feeb27 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:07.927717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ff9f5b-d856-422c-b182-e3aa30feeb27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:07.986014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:07.993344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff9f5b-d856-422c-b182-e3aa30feeb27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:07.994772 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ff9f5b-d856-422c-b182-e3aa30feeb27 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:08.089714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 544/1104] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:31:07 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:31:08.097530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca99a3e-bfa0-4f2f-b0cf-61d0594368cb None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:08.185862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:08.208097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca99a3e-bfa0-4f2f-b0cf-61d0594368cb None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:08.239991 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffe6b55-c6f7-44b7-90d6-cec6cf8834e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:08.240610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe6b55-c6f7-44b7-90d6-cec6cf8834e1 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:08.240908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe6b55-c6f7-44b7-90d6-cec6cf8834e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:08.241173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffe6b55-c6f7-44b7-90d6-cec6cf8834e1 None admin] PATH_INFO: `/v3/projects/c0e94ffe9f09419caf394da7273faf9e` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:08.242253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe6b55-c6f7-44b7-90d6-cec6cf8834e1 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e94ffe9f09419caf394da7273faf9e)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:08.243650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffe6b55-c6f7-44b7-90d6-cec6cf8834e1 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:08.245275 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 561/1105] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:08 2022] GET /identity/v3/projects/c0e94ffe9f09419caf394da7273faf9e => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:08.251351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff9f5b-d856-422c-b182-e3aa30feeb27 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:08.341898 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:08.347941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ff9f5b-d856-422c-b182-e3aa30feeb27 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:08.381641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51135abe-1df8-4c1a-8bea-b33cf585c7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:08.382233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51135abe-1df8-4c1a-8bea-b33cf585c7b8 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:08.382505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51135abe-1df8-4c1a-8bea-b33cf585c7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:08.382768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51135abe-1df8-4c1a-8bea-b33cf585c7b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:08.383474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51135abe-1df8-4c1a-8bea-b33cf585c7b8 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=c0e94ffe9f09419caf394da7273faf9e, user.id=99c61cd7ff3a41e49998918f554ce9fa) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:08.384037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51135abe-1df8-4c1a-8bea-b33cf585c7b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:08.385487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51135abe-1df8-4c1a-8bea-b33cf585c7b8 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:08.390957 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 545/1106] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:31:08 2022] GET /identity/v3/role_assignments?user.id=99c61cd7ff3a41e49998918f554ce9fa&scope.project.id=c0e94ffe9f09419caf394da7273faf9e&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:10.561839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae7c8-9c1d-4db6-9489-610d703ddf03 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:10.562365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae7c8-9c1d-4db6-9489-610d703ddf03 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:10.562365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0ae7c8-9c1d-4db6-9489-610d703ddf03 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:10.563704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 562/1107] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:10.569538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11b2b8-43d2-41f4-ad41-09633ecade4e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:10.569937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11b2b8-43d2-41f4-ad41-09633ecade4e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:10.570085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a11b2b8-43d2-41f4-ad41-09633ecade4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:10.592252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7a11b2b8-43d2-41f4-ad41-09633ecade4e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:10.593469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7a11b2b8-43d2-41f4-ad41-09633ecade4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:10.662690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 546/1108] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:10.936520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ae7c8-9c1d-4db6-9489-610d703ddf03 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:11.033618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0ae7c8-9c1d-4db6-9489-610d703ddf03 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:11.066225 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b9bb21-562d-4905-9ffc-a7ffdf849e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:11.066827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9bb21-562d-4905-9ffc-a7ffdf849e39 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:11.067099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9bb21-562d-4905-9ffc-a7ffdf849e39 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:11.067361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b9bb21-562d-4905-9ffc-a7ffdf849e39 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:11.068446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9bb21-562d-4905-9ffc-a7ffdf849e39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:11.069874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b9bb21-562d-4905-9ffc-a7ffdf849e39 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:11.071386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 563/1109] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:31:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:11.077079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11b2b8-43d2-41f4-ad41-09633ecade4e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:11.169861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:11.179035 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a11b2b8-43d2-41f4-ad41-09633ecade4e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:11.211964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d40a49-aedd-49f5-8906-794497ebf74f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:11.212489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d40a49-aedd-49f5-8906-794497ebf74f None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:11.212782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d40a49-aedd-49f5-8906-794497ebf74f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:11.213056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d40a49-aedd-49f5-8906-794497ebf74f None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:11.213701 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d40a49-aedd-49f5-8906-794497ebf74f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:11.215102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d40a49-aedd-49f5-8906-794497ebf74f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:11.240110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 547/1110] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:31:11 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:31:13.404682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff6b71-e1c7-4d54-8610-40ebe7cbf34d None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:13.405256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff6b71-e1c7-4d54-8610-40ebe7cbf34d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:13.405256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ff6b71-e1c7-4d54-8610-40ebe7cbf34d None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:13.406586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 564/1111] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:13.412450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb14d1-c375-46c0-9620-4908be972db0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:13.412873 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb14d1-c375-46c0-9620-4908be972db0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:13.413031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cb14d1-c375-46c0-9620-4908be972db0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:13.431629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b7cb14d1-c375-46c0-9620-4908be972db0 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:13.432835 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b7cb14d1-c375-46c0-9620-4908be972db0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:13.499311 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 548/1112] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:13.775781 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff6b71-e1c7-4d54-8610-40ebe7cbf34d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:13.791237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:13.873353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ff6b71-e1c7-4d54-8610-40ebe7cbf34d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:13.907539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d006d-a876-415b-ab30-45bea357c34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:13.908152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d006d-a876-415b-ab30-45bea357c34e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:13.908427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d006d-a876-415b-ab30-45bea357c34e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:13.908695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d006d-a876-415b-ab30-45bea357c34e None admin] PATH_INFO: `/v3/roles/952084593c184a78a952e1797a0b6c6b` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:13.910639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d006d-a876-415b-ab30-45bea357c34e None admin] RBAC: Authorizing `identity:get_role(role_id=952084593c184a78a952e1797a0b6c6b)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:13.911987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d006d-a876-415b-ab30-45bea357c34e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:13.913313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 565/1113] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:13 2022] GET /identity/v3/roles/952084593c184a78a952e1797a0b6c6b => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:13.919004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb14d1-c375-46c0-9620-4908be972db0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:13.996260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:14.020907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cb14d1-c375-46c0-9620-4908be972db0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:14.054228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52919523-e8c8-4e8c-b9fd-c8652f30ce2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:14.054790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52919523-e8c8-4e8c-b9fd-c8652f30ce2b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:14.055067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52919523-e8c8-4e8c-b9fd-c8652f30ce2b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:14.055339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52919523-e8c8-4e8c-b9fd-c8652f30ce2b None admin] PATH_INFO: `/v3/users/1ee0668651514d1cb1b23467e6c6e787` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:14.069070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52919523-e8c8-4e8c-b9fd-c8652f30ce2b None admin] RBAC: Authorizing `identity:get_user(user_id=1ee0668651514d1cb1b23467e6c6e787)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:14.070491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52919523-e8c8-4e8c-b9fd-c8652f30ce2b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:14.072062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 549/1114] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:13 2022] GET /identity/v3/users/1ee0668651514d1cb1b23467e6c6e787 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:14.077802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d006d-a876-415b-ab30-45bea357c34e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:14.157789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:14.176713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d006d-a876-415b-ab30-45bea357c34e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:14.210236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfa976-707e-4c64-a3cc-a2fb81808d85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:14.210817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfa976-707e-4c64-a3cc-a2fb81808d85 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:14.211093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfa976-707e-4c64-a3cc-a2fb81808d85 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:14.211358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edfa976-707e-4c64-a3cc-a2fb81808d85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:14.277041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfa976-707e-4c64-a3cc-a2fb81808d85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:14.278458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1edfa976-707e-4c64-a3cc-a2fb81808d85 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:14.355834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:14.374472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 566/1115] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:31:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:31:14.381001 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52919523-e8c8-4e8c-b9fd-c8652f30ce2b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:14.476972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52919523-e8c8-4e8c-b9fd-c8652f30ce2b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:14.509673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6f6907-0ab0-4f37-a9cb-b34c0261e79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:14.510281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f6907-0ab0-4f37-a9cb-b34c0261e79f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:14.510553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f6907-0ab0-4f37-a9cb-b34c0261e79f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:14.510819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6f6907-0ab0-4f37-a9cb-b34c0261e79f None admin] PATH_INFO: `/v3/projects/c0e94ffe9f09419caf394da7273faf9e` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:14.511910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f6907-0ab0-4f37-a9cb-b34c0261e79f None admin] RBAC: Authorizing `identity:get_project(project_id=c0e94ffe9f09419caf394da7273faf9e)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:14.513347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6f6907-0ab0-4f37-a9cb-b34c0261e79f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:14.514926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 550/1116] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:14 2022] GET /identity/v3/projects/c0e94ffe9f09419caf394da7273faf9e => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:14.520977 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfa976-707e-4c64-a3cc-a2fb81808d85 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:14.619526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edfa976-707e-4c64-a3cc-a2fb81808d85 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:14.652256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a021c368-af69-4c95-a231-4c677817cba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:14.652878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021c368-af69-4c95-a231-4c677817cba3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:14.653154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021c368-af69-4c95-a231-4c677817cba3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:14.653423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a021c368-af69-4c95-a231-4c677817cba3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:14.654132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021c368-af69-4c95-a231-4c677817cba3 None admin] RBAC: Adding query filter params (role.id=952084593c184a78a952e1797a0b6c6b, scope.project.id=c0e94ffe9f09419caf394da7273faf9e, user.id=1ee0668651514d1cb1b23467e6c6e787) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:14.654435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021c368-af69-4c95-a231-4c677817cba3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:14.655843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a021c368-af69-4c95-a231-4c677817cba3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:14.661581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 567/1117] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:31:14 2022] GET /identity/v3/role_assignments?user.id=1ee0668651514d1cb1b23467e6c6e787&scope.project.id=c0e94ffe9f09419caf394da7273faf9e&role.id=952084593c184a78a952e1797a0b6c6b => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:16.779580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c679802-86c8-40cd-9634-857df7aa5e78 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:16.780105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c679802-86c8-40cd-9634-857df7aa5e78 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:16.780105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c679802-86c8-40cd-9634-857df7aa5e78 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:16.781464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 551/1118] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:16.786722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13329b1-06b5-4e9f-a1c1-5e564bf02bf4 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:16.787121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13329b1-06b5-4e9f-a1c1-5e564bf02bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:16.787263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13329b1-06b5-4e9f-a1c1-5e564bf02bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:16.805944 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e13329b1-06b5-4e9f-a1c1-5e564bf02bf4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:16.807140 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e13329b1-06b5-4e9f-a1c1-5e564bf02bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:16.847935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:16.872983 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 568/1119] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:17.143435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c679802-86c8-40cd-9634-857df7aa5e78 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:17.146415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:17.244485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c679802-86c8-40cd-9634-857df7aa5e78 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:17.277454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3098f5-3df7-4338-8cdb-542a7809fb3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:17.278073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3098f5-3df7-4338-8cdb-542a7809fb3c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:17.278346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3098f5-3df7-4338-8cdb-542a7809fb3c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:17.278614 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3098f5-3df7-4338-8cdb-542a7809fb3c None admin] PATH_INFO: `/v3/roles/952084593c184a78a952e1797a0b6c6b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:17.279989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3098f5-3df7-4338-8cdb-542a7809fb3c None admin] RBAC: Authorizing `identity:get_role(role_id=952084593c184a78a952e1797a0b6c6b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:17.281375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3098f5-3df7-4338-8cdb-542a7809fb3c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:17.282692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 552/1120] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:17 2022] GET /identity/v3/roles/952084593c184a78a952e1797a0b6c6b => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:17.289080 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13329b1-06b5-4e9f-a1c1-5e564bf02bf4 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:17.385417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13329b1-06b5-4e9f-a1c1-5e564bf02bf4 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:17.418391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc4ccc-9221-4746-ab27-c15a9213ef74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:17.418941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc4ccc-9221-4746-ab27-c15a9213ef74 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:17.419220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc4ccc-9221-4746-ab27-c15a9213ef74 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:17.419493 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffc4ccc-9221-4746-ab27-c15a9213ef74 None admin] PATH_INFO: `/v3/users/1ee0668651514d1cb1b23467e6c6e787` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:17.420601 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc4ccc-9221-4746-ab27-c15a9213ef74 None admin] RBAC: Authorizing `identity:get_user(user_id=1ee0668651514d1cb1b23467e6c6e787)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:17.422041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffc4ccc-9221-4746-ab27-c15a9213ef74 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:17.423588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 569/1121] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:17 2022] GET /identity/v3/users/1ee0668651514d1cb1b23467e6c6e787 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:31:17.429405 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3098f5-3df7-4338-8cdb-542a7809fb3c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:17.495757 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:17.527661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3098f5-3df7-4338-8cdb-542a7809fb3c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:17.560142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97629218-ac45-4c3c-b51b-0909a9635999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:17.560781 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97629218-ac45-4c3c-b51b-0909a9635999 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:17.561064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97629218-ac45-4c3c-b51b-0909a9635999 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:17.561334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97629218-ac45-4c3c-b51b-0909a9635999 None admin] PATH_INFO: `/v3/projects/c0e94ffe9f09419caf394da7273faf9e` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:17.562439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97629218-ac45-4c3c-b51b-0909a9635999 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e94ffe9f09419caf394da7273faf9e)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:17.563852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97629218-ac45-4c3c-b51b-0909a9635999 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:17.566116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 553/1122] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:17 2022] GET /identity/v3/projects/c0e94ffe9f09419caf394da7273faf9e => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:17.572004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc4ccc-9221-4746-ab27-c15a9213ef74 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:17.618168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:17.670411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffc4ccc-9221-4746-ab27-c15a9213ef74 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:17.704614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf29e5f-0f6e-4280-9327-fe67d36e989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:17.705289 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf29e5f-0f6e-4280-9327-fe67d36e989c None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:17.705569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf29e5f-0f6e-4280-9327-fe67d36e989c None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:17.705868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf29e5f-0f6e-4280-9327-fe67d36e989c None admin] PATH_INFO: `/v3/projects/c0e94ffe9f09419caf394da7273faf9e/users/1ee0668651514d1cb1b23467e6c6e787/roles/952084593c184a78a952e1797a0b6c6b` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:17.707995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf29e5f-0f6e-4280-9327-fe67d36e989c None admin] RBAC: Authorizing `identity:create_grant(project_id=c0e94ffe9f09419caf394da7273faf9e, user_id=1ee0668651514d1cb1b23467e6c6e787, role_id=952084593c184a78a952e1797a0b6c6b)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:17.709459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf29e5f-0f6e-4280-9327-fe67d36e989c None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:17.721662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 570/1123] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:31:17 2022] PUT /identity/v3/projects/c0e94ffe9f09419caf394da7273faf9e/users/1ee0668651514d1cb1b23467e6c6e787/roles/952084593c184a78a952e1797a0b6c6b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:31:19.819237 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61029982-886d-4b90-b453-1595789ad40e None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:19.819787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61029982-886d-4b90-b453-1595789ad40e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:19.819787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61029982-886d-4b90-b453-1595789ad40e None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:19.821128 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 554/1124] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:19.826443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b64033-8fc2-4baf-8646-966c13fd435a None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:19.826846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b64033-8fc2-4baf-8646-966c13fd435a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:19.826935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b64033-8fc2-4baf-8646-966c13fd435a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:19.845576 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d8b64033-8fc2-4baf-8646-966c13fd435a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:19.847714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d8b64033-8fc2-4baf-8646-966c13fd435a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:19.913494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 571/1125] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:20.181103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61029982-886d-4b90-b453-1595789ad40e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:20.277916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61029982-886d-4b90-b453-1595789ad40e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:20.310792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dd96e-d7d7-42ef-8e5e-e1548bb1aa87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:20.311405 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dd96e-d7d7-42ef-8e5e-e1548bb1aa87 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:20.311676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dd96e-d7d7-42ef-8e5e-e1548bb1aa87 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:20.311940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423dd96e-d7d7-42ef-8e5e-e1548bb1aa87 None admin] PATH_INFO: `/v3/roles/952084593c184a78a952e1797a0b6c6b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:20.313329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423dd96e-d7d7-42ef-8e5e-e1548bb1aa87 None admin] RBAC: Authorizing `identity:get_role(role_id=952084593c184a78a952e1797a0b6c6b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:20.314675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423dd96e-d7d7-42ef-8e5e-e1548bb1aa87 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:20.316606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 555/1126] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:20 2022] GET /identity/v3/roles/952084593c184a78a952e1797a0b6c6b => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:20.322439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b64033-8fc2-4baf-8646-966c13fd435a None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:20.355941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:20.422312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b64033-8fc2-4baf-8646-966c13fd435a None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:20.456359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1933931-4ba6-45a8-9490-94fcdbc0f186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:20.457221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1933931-4ba6-45a8-9490-94fcdbc0f186 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:20.457503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1933931-4ba6-45a8-9490-94fcdbc0f186 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:20.457769 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1933931-4ba6-45a8-9490-94fcdbc0f186 None admin] PATH_INFO: `/v3/users/1ee0668651514d1cb1b23467e6c6e787` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:20.458886 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1933931-4ba6-45a8-9490-94fcdbc0f186 None admin] RBAC: Authorizing `identity:get_user(user_id=1ee0668651514d1cb1b23467e6c6e787)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:20.460291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1933931-4ba6-45a8-9490-94fcdbc0f186 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:20.461858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 572/1127] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:20 2022] GET /identity/v3/users/1ee0668651514d1cb1b23467e6c6e787 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:20.467637 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dd96e-d7d7-42ef-8e5e-e1548bb1aa87 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:20.476625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:20.567518 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423dd96e-d7d7-42ef-8e5e-e1548bb1aa87 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:20.601424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff445a7-9d90-445c-9abc-4320cfa360a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:20.602024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff445a7-9d90-445c-9abc-4320cfa360a1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:20.602296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff445a7-9d90-445c-9abc-4320cfa360a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:20.602561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff445a7-9d90-445c-9abc-4320cfa360a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:20.667429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff445a7-9d90-445c-9abc-4320cfa360a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:20.668881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff445a7-9d90-445c-9abc-4320cfa360a1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:20.676882 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:20.763767 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 556/1128] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:31:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:31:20.769675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1933931-4ba6-45a8-9490-94fcdbc0f186 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:20.865585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1933931-4ba6-45a8-9490-94fcdbc0f186 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:20.867052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:20.909447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae35ba09-813a-4364-a685-51b1ea104b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:20.910063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35ba09-813a-4364-a685-51b1ea104b73 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:20.910336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35ba09-813a-4364-a685-51b1ea104b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:20.910604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae35ba09-813a-4364-a685-51b1ea104b73 None admin] PATH_INFO: `/v3/projects/c0e94ffe9f09419caf394da7273faf9e` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:20.911701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35ba09-813a-4364-a685-51b1ea104b73 None admin] RBAC: Authorizing `identity:get_project(project_id=c0e94ffe9f09419caf394da7273faf9e)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:20.913150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae35ba09-813a-4364-a685-51b1ea104b73 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:20.914779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 573/1129] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:20 2022] GET /identity/v3/projects/c0e94ffe9f09419caf394da7273faf9e => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:20.921036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff445a7-9d90-445c-9abc-4320cfa360a1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:21.017837 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff445a7-9d90-445c-9abc-4320cfa360a1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:21.024873 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:21.051439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824ca94-a630-405c-b9f4-b7a670844b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:21.052030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824ca94-a630-405c-b9f4-b7a670844b0a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:21.052302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824ca94-a630-405c-b9f4-b7a670844b0a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:21.052568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824ca94-a630-405c-b9f4-b7a670844b0a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:21.053307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824ca94-a630-405c-b9f4-b7a670844b0a None admin] RBAC: Adding query filter params (role.id=952084593c184a78a952e1797a0b6c6b, scope.project.id=c0e94ffe9f09419caf394da7273faf9e, user.id=1ee0668651514d1cb1b23467e6c6e787) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:21.053607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824ca94-a630-405c-b9f4-b7a670844b0a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:21.055014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824ca94-a630-405c-b9f4-b7a670844b0a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:21.060521 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 557/1130] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:31:20 2022] GET /identity/v3/role_assignments?user.id=1ee0668651514d1cb1b23467e6c6e787&scope.project.id=c0e94ffe9f09419caf394da7273faf9e&role.id=952084593c184a78a952e1797a0b6c6b => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:23.236492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe15f6-081a-42a3-a642-1c77f30dd47f None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:23.237147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe15f6-081a-42a3-a642-1c77f30dd47f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:23.237147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfe15f6-081a-42a3-a642-1c77f30dd47f None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:23.239189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 574/1131] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:23.244377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c774875-120d-4718-90da-46f324418392 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:23.244771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c774875-120d-4718-90da-46f324418392 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:23.244948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c774875-120d-4718-90da-46f324418392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:23.262800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7c774875-120d-4718-90da-46f324418392 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:23.264002 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7c774875-120d-4718-90da-46f324418392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:23.329398 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 558/1132] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:23.603601 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe15f6-081a-42a3-a642-1c77f30dd47f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:23.702731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfe15f6-081a-42a3-a642-1c77f30dd47f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:23.734930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd59714-33f6-4e19-83d2-59821dcca41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:23.735545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59714-33f6-4e19-83d2-59821dcca41d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:23.735821 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59714-33f6-4e19-83d2-59821dcca41d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:23.736090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd59714-33f6-4e19-83d2-59821dcca41d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:23.737740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd59714-33f6-4e19-83d2-59821dcca41d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:23.739153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd59714-33f6-4e19-83d2-59821dcca41d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:23.740693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 575/1133] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:31:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:23.746647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c774875-120d-4718-90da-46f324418392 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:23.848738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:23.851804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c774875-120d-4718-90da-46f324418392 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:23.896039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec39f1c-e530-4484-92d1-4504b35f6380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:23.896645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec39f1c-e530-4484-92d1-4504b35f6380 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:23.896953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec39f1c-e530-4484-92d1-4504b35f6380 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:23.897225 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec39f1c-e530-4484-92d1-4504b35f6380 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:23.897877 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec39f1c-e530-4484-92d1-4504b35f6380 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:23.899302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec39f1c-e530-4484-92d1-4504b35f6380 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:23.925366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 559/1134] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:31:23 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:31:26.103470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a88bf-3ab5-425c-9148-6cb71d95edee None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:26.104026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a88bf-3ab5-425c-9148-6cb71d95edee None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:26.104026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490a88bf-3ab5-425c-9148-6cb71d95edee None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:26.105378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 576/1135] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:26.110466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffa098-48df-44e8-8e39-0c8f5182216a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:26.110878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffa098-48df-44e8-8e39-0c8f5182216a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:26.111019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffa098-48df-44e8-8e39-0c8f5182216a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:26.130109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-37ffa098-48df-44e8-8e39-0c8f5182216a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:26.131288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-37ffa098-48df-44e8-8e39-0c8f5182216a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:26.197316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 560/1136] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:26.478675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a88bf-3ab5-425c-9148-6cb71d95edee None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:26.521313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:26.586121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490a88bf-3ab5-425c-9148-6cb71d95edee None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:26.618908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b0ab0e-5e3f-4595-967d-93a0e3ca61e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:26.619519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0ab0e-5e3f-4595-967d-93a0e3ca61e7 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:26.619795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0ab0e-5e3f-4595-967d-93a0e3ca61e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:26.620061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b0ab0e-5e3f-4595-967d-93a0e3ca61e7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:26.621194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0ab0e-5e3f-4595-967d-93a0e3ca61e7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:26.622614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b0ab0e-5e3f-4595-967d-93a0e3ca61e7 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:26.624160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 577/1137] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:31:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:26.630527 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffa098-48df-44e8-8e39-0c8f5182216a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:26.708365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:26.726348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffa098-48df-44e8-8e39-0c8f5182216a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:26.759678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae71718d-18ae-4fb4-8330-40a2f56247d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:26.760187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71718d-18ae-4fb4-8330-40a2f56247d1 None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:26.760464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71718d-18ae-4fb4-8330-40a2f56247d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:26.760733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae71718d-18ae-4fb4-8330-40a2f56247d1 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:26.761393 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71718d-18ae-4fb4-8330-40a2f56247d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:26.762807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae71718d-18ae-4fb4-8330-40a2f56247d1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:26.782550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 561/1138] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:31:26 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:31:28.980375 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459b7af-ce9d-49aa-aaf6-b6779e6bd39f None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:28.980985 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459b7af-ce9d-49aa-aaf6-b6779e6bd39f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:28.980985 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3459b7af-ce9d-49aa-aaf6-b6779e6bd39f None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:28.982295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 578/1139] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:28.987611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5a3b-5637-4db8-b95f-86a06ef69afd None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:28.988547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5a3b-5637-4db8-b95f-86a06ef69afd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:28.988861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187d5a3b-5637-4db8-b95f-86a06ef69afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:29.007265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-187d5a3b-5637-4db8-b95f-86a06ef69afd None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:29.008451 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-187d5a3b-5637-4db8-b95f-86a06ef69afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:29.073894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 562/1140] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:29.344776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459b7af-ce9d-49aa-aaf6-b6779e6bd39f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:29.442584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3459b7af-ce9d-49aa-aaf6-b6779e6bd39f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:29.449659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:29.476652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868c3c4-927e-4443-b5a0-9b363d3bd045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:29.477328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868c3c4-927e-4443-b5a0-9b363d3bd045 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:29.477605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868c3c4-927e-4443-b5a0-9b363d3bd045 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:29.477876 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a868c3c4-927e-4443-b5a0-9b363d3bd045 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:29.487600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a868c3c4-927e-4443-b5a0-9b363d3bd045 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-a868c3c4-927e-4443-b5a0-9b363d3bd045 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:31:29.492627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:29.493860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:29.494797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 579/1141] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:31:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:29.499696 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d5a3b-5637-4db8-b95f-86a06ef69afd None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:29.564717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:29.599167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-187d5a3b-5637-4db8-b95f-86a06ef69afd None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:29.632267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e890e3-0af3-4ff4-a503-9565b4b95a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:29.632906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e890e3-0af3-4ff4-a503-9565b4b95a9b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:29.633186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e890e3-0af3-4ff4-a503-9565b4b95a9b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:29.633459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e890e3-0af3-4ff4-a503-9565b4b95a9b None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:29.634386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e890e3-0af3-4ff4-a503-9565b4b95a9b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:29.634686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e890e3-0af3-4ff4-a503-9565b4b95a9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:29.636050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e890e3-0af3-4ff4-a503-9565b4b95a9b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:29.643861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 563/1142] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:31:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:29.650016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868c3c4-927e-4443-b5a0-9b363d3bd045 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:29.746564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a868c3c4-927e-4443-b5a0-9b363d3bd045 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:29.779348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5e894-25bb-49dc-8431-458894e84804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:29.779952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5e894-25bb-49dc-8431-458894e84804 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:29.780227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5e894-25bb-49dc-8431-458894e84804 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:29.780495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f5e894-25bb-49dc-8431-458894e84804 None admin] PATH_INFO: `/v3/users/b84f968cb1dc45699c9fae3562fb9f41` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:29.793497 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5e894-25bb-49dc-8431-458894e84804 None admin] RBAC: Authorizing `identity:get_user(user_id=b84f968cb1dc45699c9fae3562fb9f41)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:29.794911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f5e894-25bb-49dc-8431-458894e84804 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:29.796467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 580/1143] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:29 2022] GET /identity/v3/users/b84f968cb1dc45699c9fae3562fb9f41 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:29.802793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e890e3-0af3-4ff4-a503-9565b4b95a9b None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:29.902809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e890e3-0af3-4ff4-a503-9565b4b95a9b None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:29.932015 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:29.937832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef73c21-b621-4a03-b99d-1c4a22f3b379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:29.938421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef73c21-b621-4a03-b99d-1c4a22f3b379 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:29.938697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef73c21-b621-4a03-b99d-1c4a22f3b379 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:29.938965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef73c21-b621-4a03-b99d-1c4a22f3b379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:30.004586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef73c21-b621-4a03-b99d-1c4a22f3b379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:30.006052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef73c21-b621-4a03-b99d-1c4a22f3b379 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:30.099043 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 564/1144] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:31:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:31:30.105069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5e894-25bb-49dc-8431-458894e84804 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:30.130138 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:30.202489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f5e894-25bb-49dc-8431-458894e84804 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:30.234957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c9d83f-bd2d-40df-b75e-0e1f9ca2d8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:30.235563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9d83f-bd2d-40df-b75e-0e1f9ca2d8af None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:30.235835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9d83f-bd2d-40df-b75e-0e1f9ca2d8af None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:30.236100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c9d83f-bd2d-40df-b75e-0e1f9ca2d8af None admin] PATH_INFO: `/v3/projects/7657747080004ab8b72d03e856909a61` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:30.237206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9d83f-bd2d-40df-b75e-0e1f9ca2d8af None admin] RBAC: Authorizing `identity:get_project(project_id=7657747080004ab8b72d03e856909a61)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:30.239195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42c9d83f-bd2d-40df-b75e-0e1f9ca2d8af None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:30.240957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 581/1145] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:30 2022] GET /identity/v3/projects/7657747080004ab8b72d03e856909a61 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:30.246806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef73c21-b621-4a03-b99d-1c4a22f3b379 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:30.275977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:30.343278 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef73c21-b621-4a03-b99d-1c4a22f3b379 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:30.376016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5768a5bc-ad87-4ba6-b16c-6162c9e7d9d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:30.376599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768a5bc-ad87-4ba6-b16c-6162c9e7d9d6 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:30.376914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768a5bc-ad87-4ba6-b16c-6162c9e7d9d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:30.377182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5768a5bc-ad87-4ba6-b16c-6162c9e7d9d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:30.377889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768a5bc-ad87-4ba6-b16c-6162c9e7d9d6 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=7657747080004ab8b72d03e856909a61, user.id=b84f968cb1dc45699c9fae3562fb9f41) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:30.378189 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768a5bc-ad87-4ba6-b16c-6162c9e7d9d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:30.379680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5768a5bc-ad87-4ba6-b16c-6162c9e7d9d6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:30.385093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 565/1146] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:31:30 2022] GET /identity/v3/role_assignments?user.id=b84f968cb1dc45699c9fae3562fb9f41&scope.project.id=7657747080004ab8b72d03e856909a61&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:32.487132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b46a7-8cc7-4a63-8350-2e1dc635086a None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:32.487687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b46a7-8cc7-4a63-8350-2e1dc635086a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:32.487687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b46a7-8cc7-4a63-8350-2e1dc635086a None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:32.489755 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 582/1147] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:32.494811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf69e-12af-439d-9a1e-b160aba06f47 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:32.495252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf69e-12af-439d-9a1e-b160aba06f47 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:32.495378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21cf69e-12af-439d-9a1e-b160aba06f47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:32.513287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d21cf69e-12af-439d-9a1e-b160aba06f47 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:32.514451 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d21cf69e-12af-439d-9a1e-b160aba06f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:32.578282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 566/1148] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:32.847820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b46a7-8cc7-4a63-8350-2e1dc635086a None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:32.936399 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:32.946467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b46a7-8cc7-4a63-8350-2e1dc635086a None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:32.978894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adefac-e55f-4722-8c86-bb3608d4080a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:32.979480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adefac-e55f-4722-8c86-bb3608d4080a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:32.979751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adefac-e55f-4722-8c86-bb3608d4080a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:32.980013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adefac-e55f-4722-8c86-bb3608d4080a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:32.989704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adefac-e55f-4722-8c86-bb3608d4080a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-e6adefac-e55f-4722-8c86-bb3608d4080a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:31:32.994351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:32.995442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:32.996321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 583/1149] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:31:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:33.001383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cf69e-12af-439d-9a1e-b160aba06f47 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:33.026193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:33.098630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21cf69e-12af-439d-9a1e-b160aba06f47 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:33.131515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8e651-b9af-44f2-9887-7a29699851a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:33.132099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8e651-b9af-44f2-9887-7a29699851a9 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:33.132376 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8e651-b9af-44f2-9887-7a29699851a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:33.132647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8e651-b9af-44f2-9887-7a29699851a9 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:33.133326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8e651-b9af-44f2-9887-7a29699851a9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:33.133623 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8e651-b9af-44f2-9887-7a29699851a9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:33.135228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8e651-b9af-44f2-9887-7a29699851a9 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:33.143005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 567/1150] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:31:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:33.149707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adefac-e55f-4722-8c86-bb3608d4080a None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:33.244574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adefac-e55f-4722-8c86-bb3608d4080a None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:33.277307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a50685-0a4d-4225-b66c-a0717dd32e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:33.277911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a50685-0a4d-4225-b66c-a0717dd32e8f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:33.278181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a50685-0a4d-4225-b66c-a0717dd32e8f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:33.278448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a50685-0a4d-4225-b66c-a0717dd32e8f None admin] PATH_INFO: `/v3/users/b84f968cb1dc45699c9fae3562fb9f41` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:33.279544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a50685-0a4d-4225-b66c-a0717dd32e8f None admin] RBAC: Authorizing `identity:get_user(user_id=b84f968cb1dc45699c9fae3562fb9f41)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:33.280973 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a50685-0a4d-4225-b66c-a0717dd32e8f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:33.282517 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 584/1151] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:33 2022] GET /identity/v3/users/b84f968cb1dc45699c9fae3562fb9f41 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:33.288517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8e651-b9af-44f2-9887-7a29699851a9 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:33.371438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:33.386486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8e651-b9af-44f2-9887-7a29699851a9 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:33.418216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d5351-34ad-407f-8ac3-07390f22ab0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:33.418833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d5351-34ad-407f-8ac3-07390f22ab0b None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:33.419110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d5351-34ad-407f-8ac3-07390f22ab0b None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:33.419381 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d5351-34ad-407f-8ac3-07390f22ab0b None admin] PATH_INFO: `/v3/projects/7657747080004ab8b72d03e856909a61` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:33.420470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d5351-34ad-407f-8ac3-07390f22ab0b None admin] RBAC: Authorizing `identity:get_project(project_id=7657747080004ab8b72d03e856909a61)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:33.421921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d5351-34ad-407f-8ac3-07390f22ab0b None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:33.423519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 568/1152] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:33 2022] GET /identity/v3/projects/7657747080004ab8b72d03e856909a61 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:33.430305 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a50685-0a4d-4225-b66c-a0717dd32e8f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:33.452014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:33.527467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a50685-0a4d-4225-b66c-a0717dd32e8f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:33.560428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3a4a7b-f6fb-42cb-ac10-2eb43749a066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:33.561105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a4a7b-f6fb-42cb-ac10-2eb43749a066 None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:33.561382 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a4a7b-f6fb-42cb-ac10-2eb43749a066 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:33.561650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3a4a7b-f6fb-42cb-ac10-2eb43749a066 None admin] PATH_INFO: `/v3/projects/7657747080004ab8b72d03e856909a61/users/b84f968cb1dc45699c9fae3562fb9f41/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:33.563529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a4a7b-f6fb-42cb-ac10-2eb43749a066 None admin] RBAC: Authorizing `identity:create_grant(project_id=7657747080004ab8b72d03e856909a61, user_id=b84f968cb1dc45699c9fae3562fb9f41, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:33.564974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3a4a7b-f6fb-42cb-ac10-2eb43749a066 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:33.579085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 585/1153] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:31:33 2022] PUT /identity/v3/projects/7657747080004ab8b72d03e856909a61/users/b84f968cb1dc45699c9fae3562fb9f41/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:31:35.749560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f429de-1a4e-455f-b1da-2abc2bdea9c7 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:35.750104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f429de-1a4e-455f-b1da-2abc2bdea9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:35.750104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f429de-1a4e-455f-b1da-2abc2bdea9c7 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:35.751416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 569/1154] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:35.756884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef5e6d-2c59-4bae-af6c-789b2b2ddb04 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:35.757325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef5e6d-2c59-4bae-af6c-789b2b2ddb04 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:35.757428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ef5e6d-2c59-4bae-af6c-789b2b2ddb04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:35.776138 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-44ef5e6d-2c59-4bae-af6c-789b2b2ddb04 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:35.777347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-44ef5e6d-2c59-4bae-af6c-789b2b2ddb04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:35.842192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 586/1155] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:36.141228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f429de-1a4e-455f-b1da-2abc2bdea9c7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:36.243845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f429de-1a4e-455f-b1da-2abc2bdea9c7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:36.277547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c151b4d-6f89-4c10-ad18-c4d170bba0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:36.278165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c151b4d-6f89-4c10-ad18-c4d170bba0b3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:36.278441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c151b4d-6f89-4c10-ad18-c4d170bba0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:36.278709 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c151b4d-6f89-4c10-ad18-c4d170bba0b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:36.289122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c151b4d-6f89-4c10-ad18-c4d170bba0b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-4c151b4d-6f89-4c10-ad18-c4d170bba0b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:31:36.293921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:36.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:31:36.296079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 570/1156] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:31:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:36.301988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef5e6d-2c59-4bae-af6c-789b2b2ddb04 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:36.313464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:36.403613 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ef5e6d-2c59-4bae-af6c-789b2b2ddb04 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:36.436360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f54e7d-54c1-458b-bc19-4ecf6e5be6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:36.436981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f54e7d-54c1-458b-bc19-4ecf6e5be6d6 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:36.437258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f54e7d-54c1-458b-bc19-4ecf6e5be6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:36.437528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f54e7d-54c1-458b-bc19-4ecf6e5be6d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:36.438178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f54e7d-54c1-458b-bc19-4ecf6e5be6d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:36.438478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f54e7d-54c1-458b-bc19-4ecf6e5be6d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:36.439830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f54e7d-54c1-458b-bc19-4ecf6e5be6d6 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:36.448162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 587/1157] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:31:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:36.454297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c151b4d-6f89-4c10-ad18-c4d170bba0b3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:36.470118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:36.552047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c151b4d-6f89-4c10-ad18-c4d170bba0b3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:36.585171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1b4d8-3ecf-42a6-aa79-621a64f4ce67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:36.585787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1b4d8-3ecf-42a6-aa79-621a64f4ce67 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:36.586058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1b4d8-3ecf-42a6-aa79-621a64f4ce67 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:36.586324 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1b4d8-3ecf-42a6-aa79-621a64f4ce67 None admin] PATH_INFO: `/v3/users/b84f968cb1dc45699c9fae3562fb9f41` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:36.587420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1b4d8-3ecf-42a6-aa79-621a64f4ce67 None admin] RBAC: Authorizing `identity:get_user(user_id=b84f968cb1dc45699c9fae3562fb9f41)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:36.588862 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1b4d8-3ecf-42a6-aa79-621a64f4ce67 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:36.590410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 571/1158] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:36 2022] GET /identity/v3/users/b84f968cb1dc45699c9fae3562fb9f41 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:36.596299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f54e7d-54c1-458b-bc19-4ecf6e5be6d6 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:36.667773 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:36.696850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f54e7d-54c1-458b-bc19-4ecf6e5be6d6 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:36.728716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dbb7d-b8c9-4d67-b797-52133eb165b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:36.729335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dbb7d-b8c9-4d67-b797-52133eb165b1 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:36.729611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dbb7d-b8c9-4d67-b797-52133eb165b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:36.729879 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8dbb7d-b8c9-4d67-b797-52133eb165b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:36.793923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dbb7d-b8c9-4d67-b797-52133eb165b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:36.795343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8dbb7d-b8c9-4d67-b797-52133eb165b1 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:36.859551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:36.889795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 588/1159] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:31:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:31:36.895995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1b4d8-3ecf-42a6-aa79-621a64f4ce67 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:36.974448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:36.992779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1b4d8-3ecf-42a6-aa79-621a64f4ce67 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:37.024981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d8704b-4ce5-488f-a3ea-4d5c994cd95e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:37.025592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8704b-4ce5-488f-a3ea-4d5c994cd95e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:37.025863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8704b-4ce5-488f-a3ea-4d5c994cd95e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:37.026129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d8704b-4ce5-488f-a3ea-4d5c994cd95e None admin] PATH_INFO: `/v3/projects/7657747080004ab8b72d03e856909a61` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:37.027216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8704b-4ce5-488f-a3ea-4d5c994cd95e None admin] RBAC: Authorizing `identity:get_project(project_id=7657747080004ab8b72d03e856909a61)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:37.028620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d8704b-4ce5-488f-a3ea-4d5c994cd95e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:37.030241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 572/1160] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:36 2022] GET /identity/v3/projects/7657747080004ab8b72d03e856909a61 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:37.036994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dbb7d-b8c9-4d67-b797-52133eb165b1 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:37.131439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8dbb7d-b8c9-4d67-b797-52133eb165b1 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:37.165210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83187ae8-6dbf-4904-a6d2-5dc86bf1b65d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:37.165797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83187ae8-6dbf-4904-a6d2-5dc86bf1b65d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:37.166073 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83187ae8-6dbf-4904-a6d2-5dc86bf1b65d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:37.166343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83187ae8-6dbf-4904-a6d2-5dc86bf1b65d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:37.167053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83187ae8-6dbf-4904-a6d2-5dc86bf1b65d None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=7657747080004ab8b72d03e856909a61, user.id=b84f968cb1dc45699c9fae3562fb9f41) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:37.167360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83187ae8-6dbf-4904-a6d2-5dc86bf1b65d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:37.168789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83187ae8-6dbf-4904-a6d2-5dc86bf1b65d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:37.174313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 589/1161] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:31:37 2022] GET /identity/v3/role_assignments?user.id=b84f968cb1dc45699c9fae3562fb9f41&scope.project.id=7657747080004ab8b72d03e856909a61&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:39.337826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b764e6-c520-4a52-8ab6-810f58e43144 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:39.338390 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b764e6-c520-4a52-8ab6-810f58e43144 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:39.338390 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b764e6-c520-4a52-8ab6-810f58e43144 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:39.339709 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 573/1162] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:39.344785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc06ee-ad09-4e2f-a8b5-0d027347fbd5 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:39.345166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc06ee-ad09-4e2f-a8b5-0d027347fbd5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:39.345340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edc06ee-ad09-4e2f-a8b5-0d027347fbd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:39.364466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-4edc06ee-ad09-4e2f-a8b5-0d027347fbd5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:39.365689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4edc06ee-ad09-4e2f-a8b5-0d027347fbd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:39.393502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:39.431631 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 590/1163] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:39.698973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b764e6-c520-4a52-8ab6-810f58e43144 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:39.799092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b764e6-c520-4a52-8ab6-810f58e43144 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:39.832415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ab596-c01a-42a4-8148-7830ea157f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:39.833033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ab596-c01a-42a4-8148-7830ea157f5e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:39.833315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ab596-c01a-42a4-8148-7830ea157f5e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:39.833589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6ab596-c01a-42a4-8148-7830ea157f5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:39.851777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:39.899465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ab596-c01a-42a4-8148-7830ea157f5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:39.900928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6ab596-c01a-42a4-8148-7830ea157f5e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:40.013558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 574/1164] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:31:39 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:31:40.019873 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc06ee-ad09-4e2f-a8b5-0d027347fbd5 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:40.116152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edc06ee-ad09-4e2f-a8b5-0d027347fbd5 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:40.149757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52113339-9ab9-4285-bec2-29d720c4b762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:40.150367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52113339-9ab9-4285-bec2-29d720c4b762 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:40.150641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52113339-9ab9-4285-bec2-29d720c4b762 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:40.150907 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52113339-9ab9-4285-bec2-29d720c4b762 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:40.157314 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52113339-9ab9-4285-bec2-29d720c4b762 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:40.158720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52113339-9ab9-4285-bec2-29d720c4b762 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-52113339-9ab9-4285-bec2-29d720c4b762 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:40.170606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:40.171911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:40.172995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 591/1165] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Sep 6 22:31:40 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:40.177906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ab596-c01a-42a4-8148-7830ea157f5e None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:40.226138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:40.275109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6ab596-c01a-42a4-8148-7830ea157f5e None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:40.308174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a38672-05dd-4ffb-8dd8-0a4723825eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:40.308790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a38672-05dd-4ffb-8dd8-0a4723825eff None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:40.309073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a38672-05dd-4ffb-8dd8-0a4723825eff None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:40.309341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a38672-05dd-4ffb-8dd8-0a4723825eff None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:40.309992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a38672-05dd-4ffb-8dd8-0a4723825eff None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:40.310269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a38672-05dd-4ffb-8dd8-0a4723825eff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:40.311619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a38672-05dd-4ffb-8dd8-0a4723825eff None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:40.318652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 575/1166] 10.222.0.39 () {56 vars in 1115 bytes} [Tue Sep 6 22:31:40 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:42.442961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c250b-1b40-46be-a5f8-718cd4405978 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:42.443522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c250b-1b40-46be-a5f8-718cd4405978 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:42.443522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63c250b-1b40-46be-a5f8-718cd4405978 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:42.444826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 592/1167] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:42.450104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd55e42-dc4e-4688-a864-9527a4d23dd5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:42.450538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd55e42-dc4e-4688-a864-9527a4d23dd5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:42.450654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd55e42-dc4e-4688-a864-9527a4d23dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:42.468658 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-dfd55e42-dc4e-4688-a864-9527a4d23dd5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:42.469852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dfd55e42-dc4e-4688-a864-9527a4d23dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:42.532932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 576/1168] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:42.800727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c250b-1b40-46be-a5f8-718cd4405978 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:42.827638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:42.900435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63c250b-1b40-46be-a5f8-718cd4405978 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:42.932250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001a210-785e-4b8d-8c83-f9cb0bc335c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:42.932864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001a210-785e-4b8d-8c83-f9cb0bc335c0 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:42.933139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001a210-785e-4b8d-8c83-f9cb0bc335c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:42.933405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001a210-785e-4b8d-8c83-f9cb0bc335c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:42.934053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001a210-785e-4b8d-8c83-f9cb0bc335c0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:42.935391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001a210-785e-4b8d-8c83-f9cb0bc335c0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:42.952573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 593/1169] 10.222.0.39 () {60 vars in 1138 bytes} [Tue Sep 6 22:31:42 2022] POST /identity/v3/domains => generated 267 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:31:45.310162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb34fb5-2067-418d-a739-20a8490a68a9 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:45.310691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb34fb5-2067-418d-a739-20a8490a68a9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:45.310691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb34fb5-2067-418d-a739-20a8490a68a9 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:45.312049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 577/1170] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:45.318258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef873cf-3514-451b-a073-66c5f21942a6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:45.318691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef873cf-3514-451b-a073-66c5f21942a6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:45.318792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef873cf-3514-451b-a073-66c5f21942a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:45.337221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-8ef873cf-3514-451b-a073-66c5f21942a6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:45.338405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8ef873cf-3514-451b-a073-66c5f21942a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:45.403030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 594/1171] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:45.687746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb34fb5-2067-418d-a739-20a8490a68a9 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:45.734411 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:45.788437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb34fb5-2067-418d-a739-20a8490a68a9 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:45.821338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50cf2b2-644d-40e0-bf78-2ff7391ab279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:45.821951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cf2b2-644d-40e0-bf78-2ff7391ab279 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:45.822231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cf2b2-644d-40e0-bf78-2ff7391ab279 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:45.822508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cf2b2-644d-40e0-bf78-2ff7391ab279 None admin] PATH_INFO: `/v3/domains/0daf40327c0b430f818d02d1f7659b7b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:45.832689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cf2b2-644d-40e0-bf78-2ff7391ab279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0daf40327c0b430f818d02d1f7659b7b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:45.834098 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50cf2b2-644d-40e0-bf78-2ff7391ab279 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:45.835651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 578/1172] 10.222.0.39 () {56 vars in 1183 bytes} [Tue Sep 6 22:31:45 2022] GET /identity/v3/domains/0daf40327c0b430f818d02d1f7659b7b => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:45.841479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef873cf-3514-451b-a073-66c5f21942a6 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:45.858852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:45.939699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef873cf-3514-451b-a073-66c5f21942a6 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:45.972996 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01006796-19c6-47b6-939e-5a6901d34df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:45.973558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01006796-19c6-47b6-939e-5a6901d34df8 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:45.973835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01006796-19c6-47b6-939e-5a6901d34df8 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:45.974103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01006796-19c6-47b6-939e-5a6901d34df8 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:45.974748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01006796-19c6-47b6-939e-5a6901d34df8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:45.976192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01006796-19c6-47b6-939e-5a6901d34df8 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:46.001989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 595/1173] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:31:45 2022] POST /identity/v3/projects => generated 361 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:31:48.236051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828ed0f-50a3-4503-9de1-5f6fb1b8b482 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:48.236620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828ed0f-50a3-4503-9de1-5f6fb1b8b482 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:48.236620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3828ed0f-50a3-4503-9de1-5f6fb1b8b482 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:48.237980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 579/1174] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:48.243998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e3aff-7ec8-4a07-a3b2-27b8ed48a4c4 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:48.244429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e3aff-7ec8-4a07-a3b2-27b8ed48a4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:48.244581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678e3aff-7ec8-4a07-a3b2-27b8ed48a4c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:48.262533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-678e3aff-7ec8-4a07-a3b2-27b8ed48a4c4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:48.263729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-678e3aff-7ec8-4a07-a3b2-27b8ed48a4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:48.322715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:48.329226 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 596/1175] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:48.600384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828ed0f-50a3-4503-9de1-5f6fb1b8b482 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:48.700440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3828ed0f-50a3-4503-9de1-5f6fb1b8b482 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:48.702670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:48.733335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e51e7-cabf-4144-889a-37d9a5408b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:48.733945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e51e7-cabf-4144-889a-37d9a5408b37 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:48.734217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e51e7-cabf-4144-889a-37d9a5408b37 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:48.734480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e51e7-cabf-4144-889a-37d9a5408b37 None admin] PATH_INFO: `/v3/domains/0daf40327c0b430f818d02d1f7659b7b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:48.735570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e51e7-cabf-4144-889a-37d9a5408b37 None admin] RBAC: Authorizing `identity:get_domain(domain_id=0daf40327c0b430f818d02d1f7659b7b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:48.736999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e51e7-cabf-4144-889a-37d9a5408b37 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:48.739066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 580/1176] 10.222.0.39 () {56 vars in 1183 bytes} [Tue Sep 6 22:31:48 2022] GET /identity/v3/domains/0daf40327c0b430f818d02d1f7659b7b => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:48.744889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e3aff-7ec8-4a07-a3b2-27b8ed48a4c4 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:48.841030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678e3aff-7ec8-4a07-a3b2-27b8ed48a4c4 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:48.875545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e39b8b-edc9-4b9d-8625-6086dd4fd085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:48.876126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e39b8b-edc9-4b9d-8625-6086dd4fd085 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:48.876398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e39b8b-edc9-4b9d-8625-6086dd4fd085 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:48.876667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e39b8b-edc9-4b9d-8625-6086dd4fd085 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:48.877334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e39b8b-edc9-4b9d-8625-6086dd4fd085 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:48.879201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e39b8b-edc9-4b9d-8625-6086dd4fd085 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:48.900494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 597/1177] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:31:48 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:31:51.058997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0951a5a-d5be-4c61-89aa-51149dcdbda3 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:51.059571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0951a5a-d5be-4c61-89aa-51149dcdbda3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:51.059571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0951a5a-d5be-4c61-89aa-51149dcdbda3 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:51.060883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 581/1178] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:51.066521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab820ae2-a03f-460b-82e4-92928d2215a0 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:51.066931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab820ae2-a03f-460b-82e4-92928d2215a0 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:51.067113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab820ae2-a03f-460b-82e4-92928d2215a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:51.085708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-ab820ae2-a03f-460b-82e4-92928d2215a0 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:51.086879 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ab820ae2-a03f-460b-82e4-92928d2215a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:51.133781 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:51.152415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 598/1179] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:51.456306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0951a5a-d5be-4c61-89aa-51149dcdbda3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:51.554938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0951a5a-d5be-4c61-89aa-51149dcdbda3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:51.588500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955a88-ddcd-45d5-a06a-e41b669b262a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:51.589142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955a88-ddcd-45d5-a06a-e41b669b262a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:51.589416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955a88-ddcd-45d5-a06a-e41b669b262a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:51.589683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89955a88-ddcd-45d5-a06a-e41b669b262a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:51.599321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89955a88-ddcd-45d5-a06a-e41b669b262a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-89955a88-ddcd-45d5-a06a-e41b669b262a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:31:51.604032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:51.605157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:31:51.606031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 582/1180] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:31:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:51.611161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab820ae2-a03f-460b-82e4-92928d2215a0 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:51.711182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab820ae2-a03f-460b-82e4-92928d2215a0 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:51.747270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353eb89f-9493-4a0b-a23c-2e804f973f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:51.747867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eb89f-9493-4a0b-a23c-2e804f973f16 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:51.748144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eb89f-9493-4a0b-a23c-2e804f973f16 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:51.748415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353eb89f-9493-4a0b-a23c-2e804f973f16 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:51.749098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353eb89f-9493-4a0b-a23c-2e804f973f16 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:51.749403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353eb89f-9493-4a0b-a23c-2e804f973f16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:51.750753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353eb89f-9493-4a0b-a23c-2e804f973f16 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:51.758599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 599/1181] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:31:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:51.764869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955a88-ddcd-45d5-a06a-e41b669b262a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:51.795782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:51.861473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89955a88-ddcd-45d5-a06a-e41b669b262a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:51.894131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffe599-337f-4a38-8475-436b7ca3a02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:51.894741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffe599-337f-4a38-8475-436b7ca3a02c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:51.895078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffe599-337f-4a38-8475-436b7ca3a02c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:51.895280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bffe599-337f-4a38-8475-436b7ca3a02c None admin] PATH_INFO: `/v3/users/8dc89d8823274e10af0d1ce859fdee47` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:51.907632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffe599-337f-4a38-8475-436b7ca3a02c None admin] RBAC: Authorizing `identity:get_user(user_id=8dc89d8823274e10af0d1ce859fdee47)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:51.909073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bffe599-337f-4a38-8475-436b7ca3a02c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:51.911236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 583/1182] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:51 2022] GET /identity/v3/users/8dc89d8823274e10af0d1ce859fdee47 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:51.917257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353eb89f-9493-4a0b-a23c-2e804f973f16 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:51.962589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:52.014144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353eb89f-9493-4a0b-a23c-2e804f973f16 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:52.046415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc630a-7198-43df-b7cd-1560787f413e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:52.046998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc630a-7198-43df-b7cd-1560787f413e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:52.047270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc630a-7198-43df-b7cd-1560787f413e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:52.047536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dc630a-7198-43df-b7cd-1560787f413e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:52.111239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc630a-7198-43df-b7cd-1560787f413e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:52.112648 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65dc630a-7198-43df-b7cd-1560787f413e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:52.154952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:52.207379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 600/1183] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:31:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:31:52.213506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffe599-337f-4a38-8475-436b7ca3a02c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:52.299914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:52.309684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bffe599-337f-4a38-8475-436b7ca3a02c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:52.341972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44d7b30-d2b5-4b90-952a-3ae2641cc20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:52.342575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d7b30-d2b5-4b90-952a-3ae2641cc20e None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:52.342845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d7b30-d2b5-4b90-952a-3ae2641cc20e None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:52.343108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44d7b30-d2b5-4b90-952a-3ae2641cc20e None admin] PATH_INFO: `/v3/projects/23902b895c02442e8ec2daa2b3c5644d` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:52.344198 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d7b30-d2b5-4b90-952a-3ae2641cc20e None admin] RBAC: Authorizing `identity:get_project(project_id=23902b895c02442e8ec2daa2b3c5644d)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:52.345630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44d7b30-d2b5-4b90-952a-3ae2641cc20e None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:52.347472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 584/1184] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:52 2022] GET /identity/v3/projects/23902b895c02442e8ec2daa2b3c5644d => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:52.353359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc630a-7198-43df-b7cd-1560787f413e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:52.448915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dc630a-7198-43df-b7cd-1560787f413e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:52.481457 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9798a8a-5e44-4c8b-88c3-0d39d66d3eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:52.481983 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9798a8a-5e44-4c8b-88c3-0d39d66d3eb8 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:52.482258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9798a8a-5e44-4c8b-88c3-0d39d66d3eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:52.482530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9798a8a-5e44-4c8b-88c3-0d39d66d3eb8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:52.483235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9798a8a-5e44-4c8b-88c3-0d39d66d3eb8 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=23902b895c02442e8ec2daa2b3c5644d, user.id=8dc89d8823274e10af0d1ce859fdee47) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:52.483539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9798a8a-5e44-4c8b-88c3-0d39d66d3eb8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:52.484966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9798a8a-5e44-4c8b-88c3-0d39d66d3eb8 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:52.490688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 601/1185] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:31:52 2022] GET /identity/v3/role_assignments?user.id=8dc89d8823274e10af0d1ce859fdee47&scope.project.id=23902b895c02442e8ec2daa2b3c5644d&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:54.630809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db929f2c-c936-496c-ad05-53003c213edb None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:54.631332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db929f2c-c936-496c-ad05-53003c213edb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:54.631332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db929f2c-c936-496c-ad05-53003c213edb None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:54.632659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 585/1186] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:54.638423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410129e8-97cc-4e20-8dda-7ffdd6607503 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:54.638798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410129e8-97cc-4e20-8dda-7ffdd6607503 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:54.638968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410129e8-97cc-4e20-8dda-7ffdd6607503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:54.661065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-410129e8-97cc-4e20-8dda-7ffdd6607503 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:54.662255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-410129e8-97cc-4e20-8dda-7ffdd6607503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:54.665015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:54.728595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 602/1187] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:55.001808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db929f2c-c936-496c-ad05-53003c213edb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:55.101852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db929f2c-c936-496c-ad05-53003c213edb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:55.133769 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc4c10-8383-47ae-a0f0-9d8eafffcc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:55.134372 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc4c10-8383-47ae-a0f0-9d8eafffcc38 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:55.134899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc4c10-8383-47ae-a0f0-9d8eafffcc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:55.135165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbc4c10-8383-47ae-a0f0-9d8eafffcc38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:55.145206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbc4c10-8383-47ae-a0f0-9d8eafffcc38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-dcbc4c10-8383-47ae-a0f0-9d8eafffcc38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:31:55.149923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:55.151003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:31:55.152179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 586/1188] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:31:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:55.157042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410129e8-97cc-4e20-8dda-7ffdd6607503 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:55.252595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410129e8-97cc-4e20-8dda-7ffdd6607503 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:55.286354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabaea50-6f8b-4ce0-aecf-4111669cbda4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:55.286943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabaea50-6f8b-4ce0-aecf-4111669cbda4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:55.287219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabaea50-6f8b-4ce0-aecf-4111669cbda4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:55.287489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabaea50-6f8b-4ce0-aecf-4111669cbda4 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:55.288137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabaea50-6f8b-4ce0-aecf-4111669cbda4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:55.288434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabaea50-6f8b-4ce0-aecf-4111669cbda4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:55.289814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eabaea50-6f8b-4ce0-aecf-4111669cbda4 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:55.297547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 603/1189] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:31:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:55.304312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc4c10-8383-47ae-a0f0-9d8eafffcc38 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:55.324268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:55.403435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbc4c10-8383-47ae-a0f0-9d8eafffcc38 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:55.436856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d4907-d5c5-460d-86e6-78f2181ad2bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:55.437472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d4907-d5c5-460d-86e6-78f2181ad2bc None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:55.437750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d4907-d5c5-460d-86e6-78f2181ad2bc None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:55.438022 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696d4907-d5c5-460d-86e6-78f2181ad2bc None admin] PATH_INFO: `/v3/users/8dc89d8823274e10af0d1ce859fdee47` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:55.439130 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d4907-d5c5-460d-86e6-78f2181ad2bc None admin] RBAC: Authorizing `identity:get_user(user_id=8dc89d8823274e10af0d1ce859fdee47)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:55.440550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696d4907-d5c5-460d-86e6-78f2181ad2bc None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:55.442199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 587/1190] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:55 2022] GET /identity/v3/users/8dc89d8823274e10af0d1ce859fdee47 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:55.447959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabaea50-6f8b-4ce0-aecf-4111669cbda4 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:55.450770 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:55.545508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eabaea50-6f8b-4ce0-aecf-4111669cbda4 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:55.577774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bac160-e24a-4277-a319-2a0d4b3245be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:55.578395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bac160-e24a-4277-a319-2a0d4b3245be None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:55.578666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bac160-e24a-4277-a319-2a0d4b3245be None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:55.578932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bac160-e24a-4277-a319-2a0d4b3245be None admin] PATH_INFO: `/v3/projects/23902b895c02442e8ec2daa2b3c5644d` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:55.580015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bac160-e24a-4277-a319-2a0d4b3245be None admin] RBAC: Authorizing `identity:get_project(project_id=23902b895c02442e8ec2daa2b3c5644d)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:55.581448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bac160-e24a-4277-a319-2a0d4b3245be None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:55.583123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 604/1191] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:55 2022] GET /identity/v3/projects/23902b895c02442e8ec2daa2b3c5644d => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:55.589048 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d4907-d5c5-460d-86e6-78f2181ad2bc None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:55.673202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:55.688276 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696d4907-d5c5-460d-86e6-78f2181ad2bc None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:55.720000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc688fd-c031-41e0-bdf2-7ce86e237690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:55.720635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc688fd-c031-41e0-bdf2-7ce86e237690 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:55.720946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc688fd-c031-41e0-bdf2-7ce86e237690 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:55.721215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc688fd-c031-41e0-bdf2-7ce86e237690 None admin] PATH_INFO: `/v3/projects/23902b895c02442e8ec2daa2b3c5644d/users/8dc89d8823274e10af0d1ce859fdee47/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:55.723621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc688fd-c031-41e0-bdf2-7ce86e237690 None admin] RBAC: Authorizing `identity:create_grant(project_id=23902b895c02442e8ec2daa2b3c5644d, user_id=8dc89d8823274e10af0d1ce859fdee47, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:55.725058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc688fd-c031-41e0-bdf2-7ce86e237690 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:55.737228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 588/1192] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:31:55 2022] PUT /identity/v3/projects/23902b895c02442e8ec2daa2b3c5644d/users/8dc89d8823274e10af0d1ce859fdee47/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:31:57.861036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4e9cf-2a09-4a12-a56a-39e5d0c81d39 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:57.861647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4e9cf-2a09-4a12-a56a-39e5d0c81d39 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:57.861647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c4e9cf-2a09-4a12-a56a-39e5d0c81d39 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:57.863589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 605/1193] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:31:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:31:57.869152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b964026-8c8e-4a38-8a47-3067f14c0623 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:57.869595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b964026-8c8e-4a38-8a47-3067f14c0623 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:57.869705 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b964026-8c8e-4a38-8a47-3067f14c0623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:57.887781 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0b964026-8c8e-4a38-8a47-3067f14c0623 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:31:57.888979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0b964026-8c8e-4a38-8a47-3067f14c0623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:31:57.953056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 589/1194] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:31:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:31:58.221328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4e9cf-2a09-4a12-a56a-39e5d0c81d39 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:58.292376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:58.324832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c4e9cf-2a09-4a12-a56a-39e5d0c81d39 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:58.358838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e629c4-6743-4f5a-aed7-6d3a4822868d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:58.359449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e629c4-6743-4f5a-aed7-6d3a4822868d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:58.359722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e629c4-6743-4f5a-aed7-6d3a4822868d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:58.359989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e629c4-6743-4f5a-aed7-6d3a4822868d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:58.370191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e629c4-6743-4f5a-aed7-6d3a4822868d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-31e629c4-6743-4f5a-aed7-6d3a4822868d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:31:58.374913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:31:58.376144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:31:58.377269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 606/1195] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:31:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:31:58.382417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b964026-8c8e-4a38-8a47-3067f14c0623 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:58.457370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:58.480448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b964026-8c8e-4a38-8a47-3067f14c0623 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:58.513057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38aa2f-65ee-43be-bc13-0819dd800963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:58.513638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38aa2f-65ee-43be-bc13-0819dd800963 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:58.513908 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38aa2f-65ee-43be-bc13-0819dd800963 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:58.514174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38aa2f-65ee-43be-bc13-0819dd800963 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:58.514813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38aa2f-65ee-43be-bc13-0819dd800963 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:58.515105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38aa2f-65ee-43be-bc13-0819dd800963 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:58.516447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38aa2f-65ee-43be-bc13-0819dd800963 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:58.524815 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 590/1196] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:31:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:58.530636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e629c4-6743-4f5a-aed7-6d3a4822868d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:58.629301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e629c4-6743-4f5a-aed7-6d3a4822868d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:58.660115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:58.665949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0db070-1b72-43a7-a342-5368cbcbdbf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:58.666565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0db070-1b72-43a7-a342-5368cbcbdbf8 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:58.666841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0db070-1b72-43a7-a342-5368cbcbdbf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:58.667109 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0db070-1b72-43a7-a342-5368cbcbdbf8 None admin] PATH_INFO: `/v3/users/8dc89d8823274e10af0d1ce859fdee47` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:58.668208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0db070-1b72-43a7-a342-5368cbcbdbf8 None admin] RBAC: Authorizing `identity:get_user(user_id=8dc89d8823274e10af0d1ce859fdee47)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:58.669662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0db070-1b72-43a7-a342-5368cbcbdbf8 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:58.671222 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 607/1197] 10.222.0.39 () {56 vars in 1177 bytes} [Tue Sep 6 22:31:58 2022] GET /identity/v3/users/8dc89d8823274e10af0d1ce859fdee47 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:58.677222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38aa2f-65ee-43be-bc13-0819dd800963 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:58.772307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38aa2f-65ee-43be-bc13-0819dd800963 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:58.804164 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c236c07-6743-4e44-a894-1d03b7268b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:58.804747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c236c07-6743-4e44-a894-1d03b7268b3a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:58.805050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c236c07-6743-4e44-a894-1d03b7268b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:58.805319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c236c07-6743-4e44-a894-1d03b7268b3a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:58.813319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:58.869224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c236c07-6743-4e44-a894-1d03b7268b3a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:58.870641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c236c07-6743-4e44-a894-1d03b7268b3a None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:58.965093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 591/1198] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:31:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:31:58.971042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0db070-1b72-43a7-a342-5368cbcbdbf8 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:59.067850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0db070-1b72-43a7-a342-5368cbcbdbf8 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:59.099735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7b6cc-7d37-4c0d-994c-cdc3772edcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:59.100336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7b6cc-7d37-4c0d-994c-cdc3772edcc9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:59.100606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7b6cc-7d37-4c0d-994c-cdc3772edcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:59.100899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7b6cc-7d37-4c0d-994c-cdc3772edcc9 None admin] PATH_INFO: `/v3/projects/23902b895c02442e8ec2daa2b3c5644d` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:59.101981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7b6cc-7d37-4c0d-994c-cdc3772edcc9 None admin] RBAC: Authorizing `identity:get_project(project_id=23902b895c02442e8ec2daa2b3c5644d)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:59.103381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b7b6cc-7d37-4c0d-994c-cdc3772edcc9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:59.104976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 608/1199] 10.222.0.39 () {56 vars in 1186 bytes} [Tue Sep 6 22:31:58 2022] GET /identity/v3/projects/23902b895c02442e8ec2daa2b3c5644d => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:31:59.110951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c236c07-6743-4e44-a894-1d03b7268b3a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:31:59.150149 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:31:59.208671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c236c07-6743-4e44-a894-1d03b7268b3a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:31:59.241424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dda9a8-6c26-49a0-a152-06f13bfd5fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:31:59.242005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dda9a8-6c26-49a0-a152-06f13bfd5fce None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:31:59.242275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dda9a8-6c26-49a0-a152-06f13bfd5fce None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:31:59.242538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dda9a8-6c26-49a0-a152-06f13bfd5fce None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:31:59.243249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dda9a8-6c26-49a0-a152-06f13bfd5fce None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=23902b895c02442e8ec2daa2b3c5644d, user.id=8dc89d8823274e10af0d1ce859fdee47) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:31:59.243546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dda9a8-6c26-49a0-a152-06f13bfd5fce None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:31:59.244968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dda9a8-6c26-49a0-a152-06f13bfd5fce None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:31:59.250447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 592/1200] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:31:59 2022] GET /identity/v3/role_assignments?user.id=8dc89d8823274e10af0d1ce859fdee47&scope.project.id=23902b895c02442e8ec2daa2b3c5644d&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:32:09.095622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=67 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:32:09.247924 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=49 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:32:55.565161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92505cd3-e212-42b7-a98c-287382d0cfd8 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:32:55.566274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92505cd3-e212-42b7-a98c-287382d0cfd8 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:32:55.566274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92505cd3-e212-42b7-a98c-287382d0cfd8 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:32:55.567595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 609/1201] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:32:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:32:55.574047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b68008-2821-43b4-9976-ef9a9470168e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:32:55.574538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b68008-2821-43b4-9976-ef9a9470168e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:32:55.574627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b68008-2821-43b4-9976-ef9a9470168e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:32:55.593248 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a2b68008-2821-43b4-9976-ef9a9470168e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:32:55.594450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a2b68008-2821-43b4-9976-ef9a9470168e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:32:55.678255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 593/1202] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:32:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:32:56.002475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92505cd3-e212-42b7-a98c-287382d0cfd8 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:32:56.147508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92505cd3-e212-42b7-a98c-287382d0cfd8 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:32:56.202582 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c834104-b83d-4f42-97a6-7e641e89a498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:32:56.203302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c834104-b83d-4f42-97a6-7e641e89a498 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:32:56.203555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c834104-b83d-4f42-97a6-7e641e89a498 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:32:56.203722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c834104-b83d-4f42-97a6-7e641e89a498 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:32:56.216516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c834104-b83d-4f42-97a6-7e641e89a498 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:32:56.218002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c834104-b83d-4f42-97a6-7e641e89a498 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-3c834104-b83d-4f42-97a6-7e641e89a498 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:32:56.233890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:32:56.235388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:32:56.237221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:32:56.237221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:32:56.237221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:32:56.237221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 610/1203] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:32:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:32:56.245381 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b68008-2821-43b4-9976-ef9a9470168e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:32:56.367396 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b68008-2821-43b4-9976-ef9a9470168e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:32:56.409391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:32:56.418800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7650f9ee-7906-4ac0-9489-855093048b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:32:56.419414 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f9ee-7906-4ac0-9489-855093048b33 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:32:56.419690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f9ee-7906-4ac0-9489-855093048b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:32:56.419957 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7650f9ee-7906-4ac0-9489-855093048b33 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:32:56.422037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650f9ee-7906-4ac0-9489-855093048b33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:32:56.422481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650f9ee-7906-4ac0-9489-855093048b33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:32:56.426106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7650f9ee-7906-4ac0-9489-855093048b33 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:32:56.438519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 594/1204] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:32:56.444861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c834104-b83d-4f42-97a6-7e641e89a498 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:32:56.514678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:32:56.586475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c834104-b83d-4f42-97a6-7e641e89a498 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:32:56.624665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8beb3fe-7593-4816-aa67-5cd19c5b2ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:32:56.625229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beb3fe-7593-4816-aa67-5cd19c5b2ff6 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:32:56.625505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beb3fe-7593-4816-aa67-5cd19c5b2ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:32:56.625777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8beb3fe-7593-4816-aa67-5cd19c5b2ff6 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:32:56.626424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8beb3fe-7593-4816-aa67-5cd19c5b2ff6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:32:56.627856 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8beb3fe-7593-4816-aa67-5cd19c5b2ff6 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:32:56.653941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 611/1205] 10.222.0.39 () {60 vars in 1132 bytes} [Tue Sep 6 22:32:56 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:32:59.253902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601fdd5-426f-4ed2-84b3-7f063e19cae1 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:32:59.255247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601fdd5-426f-4ed2-84b3-7f063e19cae1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:32:59.255247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601fdd5-426f-4ed2-84b3-7f063e19cae1 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:32:59.255960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 595/1206] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:32:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:32:59.263309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df174a-54b9-4ce6-bd16-be921b8ad86d None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:32:59.263789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df174a-54b9-4ce6-bd16-be921b8ad86d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:32:59.263876 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df174a-54b9-4ce6-bd16-be921b8ad86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:32:59.289822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e3df174a-54b9-4ce6-bd16-be921b8ad86d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:32:59.291042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e3df174a-54b9-4ce6-bd16-be921b8ad86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:32:59.371763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 612/1207] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:32:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:32:59.653606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601fdd5-426f-4ed2-84b3-7f063e19cae1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:32:59.786970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601fdd5-426f-4ed2-84b3-7f063e19cae1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:32:59.824611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f792ec2-c06f-402c-95d1-47e94c7b58ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:32:59.825369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f792ec2-c06f-402c-95d1-47e94c7b58ef None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:32:59.825598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f792ec2-c06f-402c-95d1-47e94c7b58ef None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:32:59.825867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f792ec2-c06f-402c-95d1-47e94c7b58ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:32:59.836104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f792ec2-c06f-402c-95d1-47e94c7b58ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-6f792ec2-c06f-402c-95d1-47e94c7b58ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:32:59.840983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:32:59.842021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:32:59.842939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:32:59.845087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 596/1208] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:32:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:32:59.848408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df174a-54b9-4ce6-bd16-be921b8ad86d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:32:59.889573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:32:59.961146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df174a-54b9-4ce6-bd16-be921b8ad86d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:00.003959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e790b-7ee2-4b9f-bce9-e3c0d86a7772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:00.003959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e790b-7ee2-4b9f-bce9-e3c0d86a7772 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:00.003959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e790b-7ee2-4b9f-bce9-e3c0d86a7772 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:00.004348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e790b-7ee2-4b9f-bce9-e3c0d86a7772 None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:00.005013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e790b-7ee2-4b9f-bce9-e3c0d86a7772 None admin] RBAC: Adding query filter params (name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:00.005358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e790b-7ee2-4b9f-bce9-e3c0d86a7772 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:00.006771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e790b-7ee2-4b9f-bce9-e3c0d86a7772 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:00.016571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 613/1209] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:32:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:00.022354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f792ec2-c06f-402c-95d1-47e94c7b58ef None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:00.080863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:00.136652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f792ec2-c06f-402c-95d1-47e94c7b58ef None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:00.177386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7670f-6cd5-449a-b145-161cabb803a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:00.178004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7670f-6cd5-449a-b145-161cabb803a5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:00.178334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7670f-6cd5-449a-b145-161cabb803a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:00.178625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a7670f-6cd5-449a-b145-161cabb803a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:00.189554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7670f-6cd5-449a-b145-161cabb803a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:00.191871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a7670f-6cd5-449a-b145-161cabb803a5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-05a7670f-6cd5-449a-b145-161cabb803a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:00.207681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:00.208721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:00.209958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:00.209958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:00.209958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:00.212992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 597/1210] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:33:00.214620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e790b-7ee2-4b9f-bce9-e3c0d86a7772 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:00.320483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:00.323696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e790b-7ee2-4b9f-bce9-e3c0d86a7772 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:00.369513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547acf3-02f7-4541-b344-a34da8c396a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:00.370124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547acf3-02f7-4541-b344-a34da8c396a4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:00.370402 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547acf3-02f7-4541-b344-a34da8c396a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:00.370670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547acf3-02f7-4541-b344-a34da8c396a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:00.371309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547acf3-02f7-4541-b344-a34da8c396a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:00.371608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547acf3-02f7-4541-b344-a34da8c396a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:00.373031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547acf3-02f7-4541-b344-a34da8c396a4 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:00.388332 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 614/1211] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:00.394440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7670f-6cd5-449a-b145-161cabb803a5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:00.518690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a7670f-6cd5-449a-b145-161cabb803a5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:00.524288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:00.562907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10b3fa-886a-40d4-ac8e-c6c3d4c41c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:00.563952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10b3fa-886a-40d4-ac8e-c6c3d4c41c60 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:00.564351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10b3fa-886a-40d4-ac8e-c6c3d4c41c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:00.564678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab10b3fa-886a-40d4-ac8e-c6c3d4c41c60 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:00.577702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10b3fa-886a-40d4-ac8e-c6c3d4c41c60 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:00.579398 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab10b3fa-886a-40d4-ac8e-c6c3d4c41c60 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-ab10b3fa-886a-40d4-ac8e-c6c3d4c41c60 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:33:00.593074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:00.594251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:00.595230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:00.595230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:00.595230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:33:00.595230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:33:00.595230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:33:00.595230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:33:00.595230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:33:00.595230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:33:00.595230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:00.595230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:00.595230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 598/1212] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:00.601169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547acf3-02f7-4541-b344-a34da8c396a4 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:00.708393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547acf3-02f7-4541-b344-a34da8c396a4 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:00.744997 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd8a44-5569-4a74-8e54-e4d507f81de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:00.745600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd8a44-5569-4a74-8e54-e4d507f81de4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:00.745874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd8a44-5569-4a74-8e54-e4d507f81de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:00.746158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd8a44-5569-4a74-8e54-e4d507f81de4 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:00.746863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd8a44-5569-4a74-8e54-e4d507f81de4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:00.747165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd8a44-5569-4a74-8e54-e4d507f81de4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:00.748585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd8a44-5569-4a74-8e54-e4d507f81de4 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:00.767352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 615/1213] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Sep 6 22:33:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:00.773682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10b3fa-886a-40d4-ac8e-c6c3d4c41c60 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:00.882232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab10b3fa-886a-40d4-ac8e-c6c3d4c41c60 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:00.919927 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd4b29-e945-49bf-8991-f56517005144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:00.920533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd4b29-e945-49bf-8991-f56517005144 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:00.920835 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd4b29-e945-49bf-8991-f56517005144 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:00.921109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fd4b29-e945-49bf-8991-f56517005144 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:00.952915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:01.001757 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd4b29-e945-49bf-8991-f56517005144 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:01.003806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fd4b29-e945-49bf-8991-f56517005144 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:01.113758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 599/1214] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:33:00 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:33:01.119818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd8a44-5569-4a74-8e54-e4d507f81de4 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:01.140909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:01.232547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd8a44-5569-4a74-8e54-e4d507f81de4 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:01.268406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0904eb-a89e-44cc-ab70-4577f1b8e5dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:01.269482 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0904eb-a89e-44cc-ab70-4577f1b8e5dd None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:01.269745 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0904eb-a89e-44cc-ab70-4577f1b8e5dd None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:01.270013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0904eb-a89e-44cc-ab70-4577f1b8e5dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:01.279874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0904eb-a89e-44cc-ab70-4577f1b8e5dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:01.281536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0904eb-a89e-44cc-ab70-4577f1b8e5dd None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-6b0904eb-a89e-44cc-ab70-4577f1b8e5dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:01.298887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:01.300282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:01.301543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:01.301543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:01.301543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:01.301543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 616/1215] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:01.306305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd4b29-e945-49bf-8991-f56517005144 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:01.383203 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:01.431284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fd4b29-e945-49bf-8991-f56517005144 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:01.469940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d3430-b32f-4220-a897-532bd1663455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:01.470669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3430-b32f-4220-a897-532bd1663455 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:01.470956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3430-b32f-4220-a897-532bd1663455 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:01.471318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673d3430-b32f-4220-a897-532bd1663455 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:01.472032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d3430-b32f-4220-a897-532bd1663455 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:01.472374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d3430-b32f-4220-a897-532bd1663455 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:01.473981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673d3430-b32f-4220-a897-532bd1663455 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:01.486348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 600/1216] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:01.493009 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0904eb-a89e-44cc-ab70-4577f1b8e5dd None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:01.569917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:01.605039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0904eb-a89e-44cc-ab70-4577f1b8e5dd None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:01.641400 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f659f5-e14d-4e39-be81-182aad7c3bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:01.642027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f659f5-e14d-4e39-be81-182aad7c3bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:01.642304 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f659f5-e14d-4e39-be81-182aad7c3bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:01.642575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f659f5-e14d-4e39-be81-182aad7c3bc7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:01.649177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f659f5-e14d-4e39-be81-182aad7c3bc7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:01.650658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f659f5-e14d-4e39-be81-182aad7c3bc7 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-d7f659f5-e14d-4e39-be81-182aad7c3bc7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:33:01.662758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:01.663883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:01.665012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:33:01.665012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:33:01.665012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:33:01.665012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:33:01.665012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:33:01.665012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:33:01.665012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:01.665012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:01.665012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 617/1217] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:33:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:01.671758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d3430-b32f-4220-a897-532bd1663455 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:01.782949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673d3430-b32f-4220-a897-532bd1663455 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:01.806041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:01.819426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d32aa-c84e-4456-b797-2a3d2d2c6f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:01.820067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d32aa-c84e-4456-b797-2a3d2d2c6f87 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:01.820289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d32aa-c84e-4456-b797-2a3d2d2c6f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:01.820556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d32aa-c84e-4456-b797-2a3d2d2c6f87 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:01.821280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d32aa-c84e-4456-b797-2a3d2d2c6f87 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:01.821575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d32aa-c84e-4456-b797-2a3d2d2c6f87 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:01.822987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27d32aa-c84e-4456-b797-2a3d2d2c6f87 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:01.840580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 601/1218] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:33:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:01.842998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f659f5-e14d-4e39-be81-182aad7c3bc7 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:01.947815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f659f5-e14d-4e39-be81-182aad7c3bc7 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:01.979969 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:01.987325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18b7670-8d6b-45e2-a1fa-4e4f8af71990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:01.988517 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b7670-8d6b-45e2-a1fa-4e4f8af71990 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:01.988860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b7670-8d6b-45e2-a1fa-4e4f8af71990 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:01.989148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18b7670-8d6b-45e2-a1fa-4e4f8af71990 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:01.989930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b7670-8d6b-45e2-a1fa-4e4f8af71990 None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=1d7e1b7dfa32448a88e0c90f10cdb2e9) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:01.990259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b7670-8d6b-45e2-a1fa-4e4f8af71990 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:01.991875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18b7670-8d6b-45e2-a1fa-4e4f8af71990 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:01.997920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 618/1219] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:33:01 2022] GET /identity/v3/role_assignments?user.id=1d7e1b7dfa32448a88e0c90f10cdb2e9&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:04.388532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ff69a-619f-4a26-8505-3308e1a86fef None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:04.389115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ff69a-619f-4a26-8505-3308e1a86fef None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:04.389204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ff69a-619f-4a26-8505-3308e1a86fef None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:04.390533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 602/1220] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:33:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:33:04.396829 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6df0c-0064-43bf-8604-561a9c6d94b8 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:04.397256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6df0c-0064-43bf-8604-561a9c6d94b8 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:04.397632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6df0c-0064-43bf-8604-561a9c6d94b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:04.416891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d4f6df0c-0064-43bf-8604-561a9c6d94b8 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:33:04.418283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d4f6df0c-0064-43bf-8604-561a9c6d94b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:33:04.494528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 619/1221] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:33:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:33:04.761215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ff69a-619f-4a26-8505-3308e1a86fef None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:04.876441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ff69a-619f-4a26-8505-3308e1a86fef None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:04.913423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f281b2-dba7-4f2a-8b46-e995d34af4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:04.914041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f281b2-dba7-4f2a-8b46-e995d34af4ea None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:04.914357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f281b2-dba7-4f2a-8b46-e995d34af4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:04.914603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f281b2-dba7-4f2a-8b46-e995d34af4ea None admin] PATH_INFO: `/v3/roles/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:04.924862 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f281b2-dba7-4f2a-8b46-e995d34af4ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-21f281b2-dba7-4f2a-8b46-e995d34af4ea None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:33:04.930576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:04.931746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:04.932804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 603/1222] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:33:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:04.937847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6df0c-0064-43bf-8604-561a9c6d94b8 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:05.053888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6df0c-0064-43bf-8604-561a9c6d94b8 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:05.078124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:05.089819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62a075-6aa4-4d19-b7b6-3aa603a3625f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:05.090425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62a075-6aa4-4d19-b7b6-3aa603a3625f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:05.090708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62a075-6aa4-4d19-b7b6-3aa603a3625f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:05.090978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62a075-6aa4-4d19-b7b6-3aa603a3625f None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:05.091632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62a075-6aa4-4d19-b7b6-3aa603a3625f None admin] RBAC: Adding query filter params (name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:05.091932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62a075-6aa4-4d19-b7b6-3aa603a3625f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:05.093328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62a075-6aa4-4d19-b7b6-3aa603a3625f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:05.104044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 620/1223] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:33:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:05.109838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f281b2-dba7-4f2a-8b46-e995d34af4ea None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:05.159267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:05.227243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f281b2-dba7-4f2a-8b46-e995d34af4ea None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:05.263688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40946aed-a12b-4fab-9a56-adb71c596a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:05.264302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40946aed-a12b-4fab-9a56-adb71c596a42 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:05.264597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40946aed-a12b-4fab-9a56-adb71c596a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:05.264948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40946aed-a12b-4fab-9a56-adb71c596a42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:05.275541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40946aed-a12b-4fab-9a56-adb71c596a42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:05.277090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40946aed-a12b-4fab-9a56-adb71c596a42 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-40946aed-a12b-4fab-9a56-adb71c596a42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:05.291848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:05.293122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:05.294407 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:05.294407 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:05.294407 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:05.297976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 604/1224] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:05.298744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62a075-6aa4-4d19-b7b6-3aa603a3625f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:05.414182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62a075-6aa4-4d19-b7b6-3aa603a3625f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:05.451806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88173c-2930-4ffd-a4e5-32131a663d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:05.452405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88173c-2930-4ffd-a4e5-32131a663d52 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:05.452682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88173c-2930-4ffd-a4e5-32131a663d52 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:05.452968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88173c-2930-4ffd-a4e5-32131a663d52 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:05.453596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88173c-2930-4ffd-a4e5-32131a663d52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:05.453928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88173c-2930-4ffd-a4e5-32131a663d52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:05.455331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88173c-2930-4ffd-a4e5-32131a663d52 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:05.467941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 621/1225] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:05.473917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40946aed-a12b-4fab-9a56-adb71c596a42 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:05.574488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:05.583655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40946aed-a12b-4fab-9a56-adb71c596a42 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:05.621401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dfb52-82dc-4d53-8be6-1452898912b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:05.622165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dfb52-82dc-4d53-8be6-1452898912b1 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:05.622579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dfb52-82dc-4d53-8be6-1452898912b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:05.622948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6dfb52-82dc-4d53-8be6-1452898912b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:05.632268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6dfb52-82dc-4d53-8be6-1452898912b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:05.633812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6dfb52-82dc-4d53-8be6-1452898912b1 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-8c6dfb52-82dc-4d53-8be6-1452898912b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:33:05.646359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:05.647552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:05.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:05.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:05.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:33:05.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:33:05.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:33:05.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:33:05.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:33:05.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:33:05.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:05.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:05.648693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 605/1226] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:05.660239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88173c-2930-4ffd-a4e5-32131a663d52 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:05.712264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:05.769659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88173c-2930-4ffd-a4e5-32131a663d52 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:05.808263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f51ee8-cdd5-43b7-9ff6-779389606601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:05.808905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f51ee8-cdd5-43b7-9ff6-779389606601 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:05.809186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f51ee8-cdd5-43b7-9ff6-779389606601 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:05.809486 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f51ee8-cdd5-43b7-9ff6-779389606601 None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:05.810441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f51ee8-cdd5-43b7-9ff6-779389606601 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:05.810749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f51ee8-cdd5-43b7-9ff6-779389606601 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:05.812686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f51ee8-cdd5-43b7-9ff6-779389606601 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:05.836040 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 622/1227] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Sep 6 22:33:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:05.841877 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dfb52-82dc-4d53-8be6-1452898912b1 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:05.958013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6dfb52-82dc-4d53-8be6-1452898912b1 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:06.002140 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a07e30-cb27-4689-a40e-c980172a8288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:06.002830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a07e30-cb27-4689-a40e-c980172a8288 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:06.003256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a07e30-cb27-4689-a40e-c980172a8288 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:06.003946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a07e30-cb27-4689-a40e-c980172a8288 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:06.016028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a07e30-cb27-4689-a40e-c980172a8288 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:06.017509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a07e30-cb27-4689-a40e-c980172a8288 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-e4a07e30-cb27-4689-a40e-c980172a8288 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:06.032257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:06.033442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:06.035484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:06.035484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:06.035484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:06.035484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 606/1228] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:06.040636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f51ee8-cdd5-43b7-9ff6-779389606601 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:06.151178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f51ee8-cdd5-43b7-9ff6-779389606601 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:06.153801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:06.190284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7f4ea-b6de-472c-862e-a961b4e47de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:06.191400 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7f4ea-b6de-472c-862e-a961b4e47de4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:06.191684 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7f4ea-b6de-472c-862e-a961b4e47de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:06.191940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc7f4ea-b6de-472c-862e-a961b4e47de4 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:06.192581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7f4ea-b6de-472c-862e-a961b4e47de4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:06.192933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7f4ea-b6de-472c-862e-a961b4e47de4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:06.194277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc7f4ea-b6de-472c-862e-a961b4e47de4 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:06.206212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 623/1229] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:06.212252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a07e30-cb27-4689-a40e-c980172a8288 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:06.223118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:06.326404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a07e30-cb27-4689-a40e-c980172a8288 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:06.367825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5226d-ee34-4c8d-9e0e-85c100407650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:06.368520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5226d-ee34-4c8d-9e0e-85c100407650 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:06.368860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5226d-ee34-4c8d-9e0e-85c100407650 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:06.369185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c5226d-ee34-4c8d-9e0e-85c100407650 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:06.376879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5226d-ee34-4c8d-9e0e-85c100407650 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:06.379336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c5226d-ee34-4c8d-9e0e-85c100407650 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-b6c5226d-ee34-4c8d-9e0e-85c100407650 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:33:06.391196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:06.392194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:06.393117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:33:06.393117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:33:06.393117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:33:06.393117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:33:06.393117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:33:06.393117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:33:06.393117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:06.393117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:06.393117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 607/1230] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:33:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:06.399668 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7f4ea-b6de-472c-862e-a961b4e47de4 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:06.501570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc7f4ea-b6de-472c-862e-a961b4e47de4 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:06.538030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047c8e10-b146-4d81-8768-6b4f603017cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:06.538627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c8e10-b146-4d81-8768-6b4f603017cf None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:06.538899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c8e10-b146-4d81-8768-6b4f603017cf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:06.539161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047c8e10-b146-4d81-8768-6b4f603017cf None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:06.539835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c8e10-b146-4d81-8768-6b4f603017cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:06.540136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c8e10-b146-4d81-8768-6b4f603017cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:06.541571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047c8e10-b146-4d81-8768-6b4f603017cf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:06.554047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 624/1231] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:33:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:06.560032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5226d-ee34-4c8d-9e0e-85c100407650 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:06.632079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:06.670065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c5226d-ee34-4c8d-9e0e-85c100407650 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:06.705941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ad4ae4-1b6b-44d9-8695-7d8f6309c0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:06.706910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad4ae4-1b6b-44d9-8695-7d8f6309c0d6 None admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:06.707046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad4ae4-1b6b-44d9-8695-7d8f6309c0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:06.707278 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ad4ae4-1b6b-44d9-8695-7d8f6309c0d6 None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/1d7e1b7dfa32448a88e0c90f10cdb2e9/roles/29088ad68ec54d74bee2847e2272e4e0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:06.725780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad4ae4-1b6b-44d9-8695-7d8f6309c0d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=1d7e1b7dfa32448a88e0c90f10cdb2e9, role_id=29088ad68ec54d74bee2847e2272e4e0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:06.727726 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ad4ae4-1b6b-44d9-8695-7d8f6309c0d6 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:06.741550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 608/1232] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:33:06 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/1d7e1b7dfa32448a88e0c90f10cdb2e9/roles/29088ad68ec54d74bee2847e2272e4e0 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:33:09.001845 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b532e-ae22-4021-a198-0fed90b029ed None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:09.002551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b532e-ae22-4021-a198-0fed90b029ed None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:09.002551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b532e-ae22-4021-a198-0fed90b029ed None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:09.004481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 625/1233] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:33:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:33:09.009929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15189d-ac27-4d48-8cef-6682e52bfcb4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:09.010386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15189d-ac27-4d48-8cef-6682e52bfcb4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:09.010496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b15189d-ac27-4d48-8cef-6682e52bfcb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:09.029961 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1b15189d-ac27-4d48-8cef-6682e52bfcb4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:33:09.031135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1b15189d-ac27-4d48-8cef-6682e52bfcb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:33:09.102854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 609/1234] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:33:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 06 22:33:09.387743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b532e-ae22-4021-a198-0fed90b029ed None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:09.417643 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:09.509186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7b532e-ae22-4021-a198-0fed90b029ed None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:09.559056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23daf491-2ccf-499a-bed4-9eca4bdc2012 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:09.560015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daf491-2ccf-499a-bed4-9eca4bdc2012 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:09.560554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daf491-2ccf-499a-bed4-9eca4bdc2012 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:09.560932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23daf491-2ccf-499a-bed4-9eca4bdc2012 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:09.574478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23daf491-2ccf-499a-bed4-9eca4bdc2012 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-23daf491-2ccf-499a-bed4-9eca4bdc2012 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:33:09.580446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:09.581906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:09.582972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 626/1235] 10.222.0.39 () {56 vars in 1102 bytes} [Tue Sep 6 22:33:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:09.588745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15189d-ac27-4d48-8cef-6682e52bfcb4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:09.652051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:09.711089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b15189d-ac27-4d48-8cef-6682e52bfcb4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:09.752227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fb78d-0ba9-4910-a122-5c29f50cc79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:09.753492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fb78d-0ba9-4910-a122-5c29f50cc79f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:09.753615 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fb78d-0ba9-4910-a122-5c29f50cc79f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:09.754031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252fb78d-0ba9-4910-a122-5c29f50cc79f None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:09.754923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fb78d-0ba9-4910-a122-5c29f50cc79f None admin] RBAC: Adding query filter params (name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:09.755229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fb78d-0ba9-4910-a122-5c29f50cc79f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:09.756987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252fb78d-0ba9-4910-a122-5c29f50cc79f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:09.766849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 610/1236] 10.222.0.39 () {56 vars in 1103 bytes} [Tue Sep 6 22:33:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:33:09.773147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23daf491-2ccf-499a-bed4-9eca4bdc2012 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:09.875093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:09.896579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23daf491-2ccf-499a-bed4-9eca4bdc2012 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:09.934973 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c95185-436b-4e66-9c45-58f2681fd97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:09.935664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c95185-436b-4e66-9c45-58f2681fd97f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:09.935945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c95185-436b-4e66-9c45-58f2681fd97f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:09.936213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c95185-436b-4e66-9c45-58f2681fd97f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:09.949047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c95185-436b-4e66-9c45-58f2681fd97f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:09.950708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c95185-436b-4e66-9c45-58f2681fd97f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-78c95185-436b-4e66-9c45-58f2681fd97f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:09.968816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:09.970008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:09.971178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:09.971178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:09.971178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:09.971178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 627/1237] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:09.977434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fb78d-0ba9-4910-a122-5c29f50cc79f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:10.094435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252fb78d-0ba9-4910-a122-5c29f50cc79f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:10.103686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:10.131866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcec976-57ef-4aeb-83c9-4a5425a0150f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:10.132600 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcec976-57ef-4aeb-83c9-4a5425a0150f None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:10.132914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcec976-57ef-4aeb-83c9-4a5425a0150f None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:10.133191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcec976-57ef-4aeb-83c9-4a5425a0150f None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:10.133985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcec976-57ef-4aeb-83c9-4a5425a0150f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:10.134323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcec976-57ef-4aeb-83c9-4a5425a0150f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:10.135865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcec976-57ef-4aeb-83c9-4a5425a0150f None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:10.151179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 611/1238] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:10.156923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c95185-436b-4e66-9c45-58f2681fd97f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:10.264829 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c95185-436b-4e66-9c45-58f2681fd97f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:10.299492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a746e9-771b-48f9-b597-22aeb6fce880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:10.300869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a746e9-771b-48f9-b597-22aeb6fce880 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:10.301249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a746e9-771b-48f9-b597-22aeb6fce880 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:10.301533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a746e9-771b-48f9-b597-22aeb6fce880 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:10.304371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:10.310282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a746e9-771b-48f9-b597-22aeb6fce880 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:10.311868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a746e9-771b-48f9-b597-22aeb6fce880 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-74a746e9-771b-48f9-b597-22aeb6fce880 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:33:10.323574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:10.324577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:10.326113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:10.326113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:10.326113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:33:10.326113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:33:10.326113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:33:10.326113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:33:10.326113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:33:10.326113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:33:10.326113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:10.326113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:10.326113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 628/1239] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:10.330539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcec976-57ef-4aeb-83c9-4a5425a0150f None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:10.436497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcec976-57ef-4aeb-83c9-4a5425a0150f None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:10.473738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439aed1-d662-4ed8-84b7-cf8686faeb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:10.474336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439aed1-d662-4ed8-84b7-cf8686faeb53 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:10.474622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439aed1-d662-4ed8-84b7-cf8686faeb53 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:10.474915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8439aed1-d662-4ed8-84b7-cf8686faeb53 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:10.475637 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439aed1-d662-4ed8-84b7-cf8686faeb53 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:10.475931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439aed1-d662-4ed8-84b7-cf8686faeb53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:10.477370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8439aed1-d662-4ed8-84b7-cf8686faeb53 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:10.497022 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 612/1240] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Sep 6 22:33:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:10.503065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a746e9-771b-48f9-b597-22aeb6fce880 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:10.623651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a746e9-771b-48f9-b597-22aeb6fce880 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:10.662709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7e82a-9efe-403e-ba73-836129ccd6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:10.663314 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7e82a-9efe-403e-ba73-836129ccd6b0 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:10.663594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7e82a-9efe-403e-ba73-836129ccd6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:10.663912 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7e82a-9efe-403e-ba73-836129ccd6b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:10.732138 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:10.741135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7e82a-9efe-403e-ba73-836129ccd6b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:10.742590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7e82a-9efe-403e-ba73-836129ccd6b0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:10.852140 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 629/1241] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:33:10 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:33:10.857945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439aed1-d662-4ed8-84b7-cf8686faeb53 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:10.894965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:10.975890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8439aed1-d662-4ed8-84b7-cf8686faeb53 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:11.026430 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87210a5-668e-4bbd-9c5a-3ee60cfbde70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:11.027047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87210a5-668e-4bbd-9c5a-3ee60cfbde70 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:11.027403 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87210a5-668e-4bbd-9c5a-3ee60cfbde70 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:11.027659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87210a5-668e-4bbd-9c5a-3ee60cfbde70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:11.038766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87210a5-668e-4bbd-9c5a-3ee60cfbde70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:11.040263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87210a5-668e-4bbd-9c5a-3ee60cfbde70 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-f87210a5-668e-4bbd-9c5a-3ee60cfbde70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:11.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:11.056449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:11.057596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:11.057596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:11.057596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:11.057596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 613/1242] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:33:11.062673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7e82a-9efe-403e-ba73-836129ccd6b0 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:11.176849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:11.183236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7e82a-9efe-403e-ba73-836129ccd6b0 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:11.220329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df560127-e9b7-4c86-86e4-1a2865beed9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:11.220968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df560127-e9b7-4c86-86e4-1a2865beed9e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:11.221240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df560127-e9b7-4c86-86e4-1a2865beed9e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:11.221507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df560127-e9b7-4c86-86e4-1a2865beed9e None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:11.222664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df560127-e9b7-4c86-86e4-1a2865beed9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:11.222963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df560127-e9b7-4c86-86e4-1a2865beed9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:11.224303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df560127-e9b7-4c86-86e4-1a2865beed9e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:11.235840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 630/1243] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:11.242832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87210a5-668e-4bbd-9c5a-3ee60cfbde70 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:11.328496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:11.353687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87210a5-668e-4bbd-9c5a-3ee60cfbde70 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:11.395380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddaeda-7a63-4134-9eb2-46b513eb34e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:11.396056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddaeda-7a63-4134-9eb2-46b513eb34e2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:11.396296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddaeda-7a63-4134-9eb2-46b513eb34e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:11.396556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddaeda-7a63-4134-9eb2-46b513eb34e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:11.403832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddaeda-7a63-4134-9eb2-46b513eb34e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:11.405464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ddaeda-7a63-4134-9eb2-46b513eb34e2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-31ddaeda-7a63-4134-9eb2-46b513eb34e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:33:11.417861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:11.418990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:11.419979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:33:11.419979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:33:11.419979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:33:11.419979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:33:11.419979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:33:11.419979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:33:11.419979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:11.419979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:11.419979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 614/1244] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:33:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:33:11.424895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df560127-e9b7-4c86-86e4-1a2865beed9e None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:11.544965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df560127-e9b7-4c86-86e4-1a2865beed9e None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:11.585155 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c17032-121c-49e6-bfb1-dff5d2e5b649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:11.585846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c17032-121c-49e6-bfb1-dff5d2e5b649 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:11.586429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c17032-121c-49e6-bfb1-dff5d2e5b649 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:11.586733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c17032-121c-49e6-bfb1-dff5d2e5b649 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:11.587666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c17032-121c-49e6-bfb1-dff5d2e5b649 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:11.588018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c17032-121c-49e6-bfb1-dff5d2e5b649 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:11.590042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c17032-121c-49e6-bfb1-dff5d2e5b649 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:11.606823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 631/1245] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:33:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:11.613070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddaeda-7a63-4134-9eb2-46b513eb34e2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:11.727687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddaeda-7a63-4134-9eb2-46b513eb34e2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:11.769565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adfd4a0-cf0f-492a-b466-31f88d410c96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:11.770166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfd4a0-cf0f-492a-b466-31f88d410c96 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:11.770437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfd4a0-cf0f-492a-b466-31f88d410c96 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:11.770700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adfd4a0-cf0f-492a-b466-31f88d410c96 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:11.771415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfd4a0-cf0f-492a-b466-31f88d410c96 None admin] RBAC: Adding query filter params (role.id=29088ad68ec54d74bee2847e2272e4e0, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=1d7e1b7dfa32448a88e0c90f10cdb2e9) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:11.771711 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfd4a0-cf0f-492a-b466-31f88d410c96 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:11.773144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adfd4a0-cf0f-492a-b466-31f88d410c96 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:11.820882 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 615/1246] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:33:11 2022] GET /identity/v3/role_assignments?user.id=1d7e1b7dfa32448a88e0c90f10cdb2e9&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=29088ad68ec54d74bee2847e2272e4e0 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:14.259599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d99a8-3ecb-4332-bbe9-8215d7b307dc None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:14.260136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d99a8-3ecb-4332-bbe9-8215d7b307dc None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:14.260136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4d99a8-3ecb-4332-bbe9-8215d7b307dc None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:14.261483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 632/1247] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:33:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:33:14.266906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef04b-b604-496a-bd98-6de88ea87b0e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:14.267319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef04b-b604-496a-bd98-6de88ea87b0e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:14.267454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef04b-b604-496a-bd98-6de88ea87b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:14.275961 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:14.290427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-241ef04b-b604-496a-bd98-6de88ea87b0e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:33:14.291814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-241ef04b-b604-496a-bd98-6de88ea87b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:33:14.363819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 616/1248] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:33:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:33:14.631585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d99a8-3ecb-4332-bbe9-8215d7b307dc None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:14.651291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:14.740006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4d99a8-3ecb-4332-bbe9-8215d7b307dc None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:14.777303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d57e9-f9c0-4065-9112-fa37ba11044a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:14.777926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d57e9-f9c0-4065-9112-fa37ba11044a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:14.778197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d57e9-f9c0-4065-9112-fa37ba11044a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:14.778459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d57e9-f9c0-4065-9112-fa37ba11044a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:14.789609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d57e9-f9c0-4065-9112-fa37ba11044a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-7c0d57e9-f9c0-4065-9112-fa37ba11044a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:33:14.794463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:14.795639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:14.796926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 633/1249] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:33:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:14.801999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ef04b-b604-496a-bd98-6de88ea87b0e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:14.908893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241ef04b-b604-496a-bd98-6de88ea87b0e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:14.940345 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:14.947124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2814201-f418-4c83-a746-bdc6c83a1203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:14.947742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2814201-f418-4c83-a746-bdc6c83a1203 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:14.948019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2814201-f418-4c83-a746-bdc6c83a1203 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:14.948287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2814201-f418-4c83-a746-bdc6c83a1203 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:14.948994 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2814201-f418-4c83-a746-bdc6c83a1203 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:14.949295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2814201-f418-4c83-a746-bdc6c83a1203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:14.950676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2814201-f418-4c83-a746-bdc6c83a1203 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:14.961429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 617/1250] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:33:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:14.967718 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d57e9-f9c0-4065-9112-fa37ba11044a None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:15.059457 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:15.084240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d57e9-f9c0-4065-9112-fa37ba11044a None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:15.119400 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10eaad-6ae4-41c9-9394-00fe185a95cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:15.120025 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10eaad-6ae4-41c9-9394-00fe185a95cf None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:15.120302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10eaad-6ae4-41c9-9394-00fe185a95cf None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:15.120571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff10eaad-6ae4-41c9-9394-00fe185a95cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:15.131365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10eaad-6ae4-41c9-9394-00fe185a95cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:15.132817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff10eaad-6ae4-41c9-9394-00fe185a95cf None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-ff10eaad-6ae4-41c9-9394-00fe185a95cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:15.147380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:15.148410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:15.149921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:15.149921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:15.149921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:15.149921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 634/1251] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 06 22:33:15.154310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2814201-f418-4c83-a746-bdc6c83a1203 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:15.260155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2814201-f418-4c83-a746-bdc6c83a1203 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:15.294436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79887-8b12-4b90-b34e-9e2a6b8f964d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:15.295032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79887-8b12-4b90-b34e-9e2a6b8f964d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:15.295311 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79887-8b12-4b90-b34e-9e2a6b8f964d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:15.295581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b79887-8b12-4b90-b34e-9e2a6b8f964d None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:15.296217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b79887-8b12-4b90-b34e-9e2a6b8f964d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:15.296518 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b79887-8b12-4b90-b34e-9e2a6b8f964d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:15.297913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b79887-8b12-4b90-b34e-9e2a6b8f964d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:15.309713 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 618/1252] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:15.315380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10eaad-6ae4-41c9-9394-00fe185a95cf None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:15.413020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff10eaad-6ae4-41c9-9394-00fe185a95cf None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:15.438045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:15.446421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224a120-b3fa-4d29-9622-397ecb5988a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:15.447033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224a120-b3fa-4d29-9622-397ecb5988a2 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:15.447306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224a120-b3fa-4d29-9622-397ecb5988a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:15.447571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1224a120-b3fa-4d29-9622-397ecb5988a2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:15.454761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224a120-b3fa-4d29-9622-397ecb5988a2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:15.456174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1224a120-b3fa-4d29-9622-397ecb5988a2 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-1224a120-b3fa-4d29-9622-397ecb5988a2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:33:15.467902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:15.469104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:15.470465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:15.470465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:15.470465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:33:15.470465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:33:15.470465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:33:15.470465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:33:15.470465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:33:15.470465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:33:15.470465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:15.470465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:15.470465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 635/1253] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:15.475453 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79887-8b12-4b90-b34e-9e2a6b8f964d None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:15.529031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:15.572451 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b79887-8b12-4b90-b34e-9e2a6b8f964d None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:15.605928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e4df2-4274-4173-b4fc-a4c37bab46d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:15.606525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e4df2-4274-4173-b4fc-a4c37bab46d2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:15.606801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e4df2-4274-4173-b4fc-a4c37bab46d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:15.607072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e4df2-4274-4173-b4fc-a4c37bab46d2 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:15.607790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e4df2-4274-4173-b4fc-a4c37bab46d2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:15.608090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e4df2-4274-4173-b4fc-a4c37bab46d2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:15.609532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e4df2-4274-4173-b4fc-a4c37bab46d2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:15.627317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 619/1254] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Sep 6 22:33:15 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:15.633389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224a120-b3fa-4d29-9622-397ecb5988a2 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:15.729941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1224a120-b3fa-4d29-9622-397ecb5988a2 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:15.761798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff321d-8926-4273-8f15-d0a859ddedb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:15.762627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff321d-8926-4273-8f15-d0a859ddedb2 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:15.762897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff321d-8926-4273-8f15-d0a859ddedb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:15.763159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fff321d-8926-4273-8f15-d0a859ddedb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:15.816486 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:15.828560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff321d-8926-4273-8f15-d0a859ddedb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:15.830016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fff321d-8926-4273-8f15-d0a859ddedb2 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:15.924557 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 636/1255] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:33:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:33:15.930405 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e4df2-4274-4173-b4fc-a4c37bab46d2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:16.036299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e4df2-4274-4173-b4fc-a4c37bab46d2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:16.048719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:16.069921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff74ab2-1094-49ab-93e6-37bced4df4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:16.070465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74ab2-1094-49ab-93e6-37bced4df4c7 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:16.070738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74ab2-1094-49ab-93e6-37bced4df4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:16.071004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff74ab2-1094-49ab-93e6-37bced4df4c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:16.080476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff74ab2-1094-49ab-93e6-37bced4df4c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:16.082268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff74ab2-1094-49ab-93e6-37bced4df4c7 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-dff74ab2-1094-49ab-93e6-37bced4df4c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:16.096546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:16.097886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:16.098968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:16.098968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:16.098968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:16.098968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 620/1256] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:16.103521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff321d-8926-4273-8f15-d0a859ddedb2 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:16.187918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:16.200180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fff321d-8926-4273-8f15-d0a859ddedb2 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:16.232192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991b1c1-d39a-4681-8c10-b33042cbaaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:16.232797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991b1c1-d39a-4681-8c10-b33042cbaaa3 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:16.233072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991b1c1-d39a-4681-8c10-b33042cbaaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:16.233337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991b1c1-d39a-4681-8c10-b33042cbaaa3 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:16.233957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991b1c1-d39a-4681-8c10-b33042cbaaa3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:16.234249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991b1c1-d39a-4681-8c10-b33042cbaaa3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:16.235579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991b1c1-d39a-4681-8c10-b33042cbaaa3 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:16.246686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 637/1257] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:16.252253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff74ab2-1094-49ab-93e6-37bced4df4c7 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:16.349623 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff74ab2-1094-49ab-93e6-37bced4df4c7 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:16.381451 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f3112-261b-4a4b-a300-eec69e21dad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:16.382056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f3112-261b-4a4b-a300-eec69e21dad2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:16.382330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f3112-261b-4a4b-a300-eec69e21dad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:16.382598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487f3112-261b-4a4b-a300-eec69e21dad2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:16.389257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f3112-261b-4a4b-a300-eec69e21dad2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:16.390673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487f3112-261b-4a4b-a300-eec69e21dad2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-487f3112-261b-4a4b-a300-eec69e21dad2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:33:16.402110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:16.403135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:16.404061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:33:16.404061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:33:16.404061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:33:16.404061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:33:16.404061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:33:16.404061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:33:16.404061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:16.404061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:16.404061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 621/1258] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:33:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:16.408950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991b1c1-d39a-4681-8c10-b33042cbaaa3 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:16.506561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991b1c1-d39a-4681-8c10-b33042cbaaa3 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:16.539822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28e1100-2a56-417c-a37d-c115628aebf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:16.540405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e1100-2a56-417c-a37d-c115628aebf4 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:16.540679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e1100-2a56-417c-a37d-c115628aebf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:16.540971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28e1100-2a56-417c-a37d-c115628aebf4 None admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:16.541643 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e1100-2a56-417c-a37d-c115628aebf4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:16.541943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e1100-2a56-417c-a37d-c115628aebf4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:16.543346 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28e1100-2a56-417c-a37d-c115628aebf4 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:16.555253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 638/1259] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:33:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:33:16.561238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f3112-261b-4a4b-a300-eec69e21dad2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:16.588193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:16.662139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487f3112-261b-4a4b-a300-eec69e21dad2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:16.695104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b360e8-b85a-4e03-9541-0500b1c75c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:16.695690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b360e8-b85a-4e03-9541-0500b1c75c50 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:16.695968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b360e8-b85a-4e03-9541-0500b1c75c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:16.696238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b360e8-b85a-4e03-9541-0500b1c75c50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:16.696975 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b360e8-b85a-4e03-9541-0500b1c75c50 None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=1d7e1b7dfa32448a88e0c90f10cdb2e9) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:16.697275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b360e8-b85a-4e03-9541-0500b1c75c50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:16.698686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b360e8-b85a-4e03-9541-0500b1c75c50 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:16.703809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 622/1260] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:33:16 2022] GET /identity/v3/role_assignments?user.id=1d7e1b7dfa32448a88e0c90f10cdb2e9&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:18.807125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8a91f-f1e8-46bb-9faf-eed76e9bc831 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:18.807639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8a91f-f1e8-46bb-9faf-eed76e9bc831 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:18.807639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c8a91f-f1e8-46bb-9faf-eed76e9bc831 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:18.809006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 639/1261] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:33:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:33:18.813982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f41544-de1f-4fc1-9391-ce1bc367aef8 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:18.814384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f41544-de1f-4fc1-9391-ce1bc367aef8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:18.814537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f41544-de1f-4fc1-9391-ce1bc367aef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:18.833918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-84f41544-de1f-4fc1-9391-ce1bc367aef8 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:33:18.835030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-84f41544-de1f-4fc1-9391-ce1bc367aef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:33:18.899842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 623/1262] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:33:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:33:19.166587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8a91f-f1e8-46bb-9faf-eed76e9bc831 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:19.178123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:19.266372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c8a91f-f1e8-46bb-9faf-eed76e9bc831 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:19.298703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c90d9-e45a-4bc0-a802-9405f804ee2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:19.299310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c90d9-e45a-4bc0-a802-9405f804ee2b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:19.299566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c90d9-e45a-4bc0-a802-9405f804ee2b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:19.299830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c90d9-e45a-4bc0-a802-9405f804ee2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:19.310196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c90d9-e45a-4bc0-a802-9405f804ee2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-cc8c90d9-e45a-4bc0-a802-9405f804ee2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:33:19.314891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:19.316865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:19.318112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 640/1263] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:33:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:19.322648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f41544-de1f-4fc1-9391-ce1bc367aef8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:19.340684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:19.420119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f41544-de1f-4fc1-9391-ce1bc367aef8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:19.453664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f4955-72ad-4b1b-8623-c0fda4a03616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:19.454258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f4955-72ad-4b1b-8623-c0fda4a03616 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:19.454536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f4955-72ad-4b1b-8623-c0fda4a03616 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:19.454806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7f4955-72ad-4b1b-8623-c0fda4a03616 None admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:19.455453 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f4955-72ad-4b1b-8623-c0fda4a03616 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:19.455748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f4955-72ad-4b1b-8623-c0fda4a03616 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:19.457176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7f4955-72ad-4b1b-8623-c0fda4a03616 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:19.465178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 624/1264] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:33:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:19.470820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c90d9-e45a-4bc0-a802-9405f804ee2b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:19.535651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:19.567905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c90d9-e45a-4bc0-a802-9405f804ee2b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:19.603690 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540f6c1-9e95-4163-8cfe-0665c762257f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:19.604304 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f6c1-9e95-4163-8cfe-0665c762257f None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:19.604577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f6c1-9e95-4163-8cfe-0665c762257f None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:19.604872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4540f6c1-9e95-4163-8cfe-0665c762257f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:19.614840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540f6c1-9e95-4163-8cfe-0665c762257f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:19.616251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4540f6c1-9e95-4163-8cfe-0665c762257f None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-4540f6c1-9e95-4163-8cfe-0665c762257f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:19.631065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:19.632191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:19.633156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:19.633156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:19.633156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:19.633156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 641/1265] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:19.638332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f4955-72ad-4b1b-8623-c0fda4a03616 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:19.720574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:19.739814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7f4955-72ad-4b1b-8623-c0fda4a03616 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:19.774275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a297aed-46f2-4504-9ba3-647704df85f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:19.774868 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a297aed-46f2-4504-9ba3-647704df85f2 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:19.775141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a297aed-46f2-4504-9ba3-647704df85f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:19.775408 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a297aed-46f2-4504-9ba3-647704df85f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:19.776032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a297aed-46f2-4504-9ba3-647704df85f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:19.776327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a297aed-46f2-4504-9ba3-647704df85f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:19.777698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a297aed-46f2-4504-9ba3-647704df85f2 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:19.789259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 625/1266] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:19.795095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540f6c1-9e95-4163-8cfe-0665c762257f None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:19.893709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4540f6c1-9e95-4163-8cfe-0665c762257f None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:19.912712 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:19.927094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d357b7-0174-494c-82d5-501172b9d0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:19.927691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d357b7-0174-494c-82d5-501172b9d0a9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:19.927963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d357b7-0174-494c-82d5-501172b9d0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:19.928224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d357b7-0174-494c-82d5-501172b9d0a9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:19.935290 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d357b7-0174-494c-82d5-501172b9d0a9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:19.936675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d357b7-0174-494c-82d5-501172b9d0a9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-74d357b7-0174-494c-82d5-501172b9d0a9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:33:19.947963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:19.949129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:19.950262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:19.950262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:19.950262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:33:19.950262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:33:19.950262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:33:19.950262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:33:19.950262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:33:19.950262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:33:19.950262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:19.950262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:19.950262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 642/1267] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:19 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:19.955329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a297aed-46f2-4504-9ba3-647704df85f2 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:20.062838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a297aed-46f2-4504-9ba3-647704df85f2 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:20.096313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432d9aa-414c-43b5-9f51-417cb42a9024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:20.096969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432d9aa-414c-43b5-9f51-417cb42a9024 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:20.097502 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432d9aa-414c-43b5-9f51-417cb42a9024 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:20.097502 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8432d9aa-414c-43b5-9f51-417cb42a9024 None admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:20.098209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432d9aa-414c-43b5-9f51-417cb42a9024 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:20.098509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432d9aa-414c-43b5-9f51-417cb42a9024 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:20.099917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8432d9aa-414c-43b5-9f51-417cb42a9024 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:20.108725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:20.118077 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 626/1268] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Sep 6 22:33:19 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:33:20.124023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d357b7-0174-494c-82d5-501172b9d0a9 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:20.221690 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d357b7-0174-494c-82d5-501172b9d0a9 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:20.254755 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4eb43-398b-4e64-a271-88a8fbd000ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:20.255353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4eb43-398b-4e64-a271-88a8fbd000ef None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:20.255631 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4eb43-398b-4e64-a271-88a8fbd000ef None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:20.255901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed4eb43-398b-4e64-a271-88a8fbd000ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:20.266273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4eb43-398b-4e64-a271-88a8fbd000ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:20.267693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed4eb43-398b-4e64-a271-88a8fbd000ef None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-bed4eb43-398b-4e64-a271-88a8fbd000ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:20.281401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:20.282359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:20.283528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:20.283528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:20.283528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:20.283528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 643/1269] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:20.288648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432d9aa-414c-43b5-9f51-417cb42a9024 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:20.391912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8432d9aa-414c-43b5-9f51-417cb42a9024 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:20.427125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23428336-4872-443a-bb66-19cce1f2f4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:20.427720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428336-4872-443a-bb66-19cce1f2f4a3 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:20.427998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428336-4872-443a-bb66-19cce1f2f4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:20.428269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23428336-4872-443a-bb66-19cce1f2f4a3 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:20.428940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23428336-4872-443a-bb66-19cce1f2f4a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:20.429241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23428336-4872-443a-bb66-19cce1f2f4a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:20.430591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23428336-4872-443a-bb66-19cce1f2f4a3 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:20.441480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 627/1270] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:20.447023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4eb43-398b-4e64-a271-88a8fbd000ef None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:20.478184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:20.545170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed4eb43-398b-4e64-a271-88a8fbd000ef None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:20.577823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0956c-8cbc-4beb-9f01-41a8c906e72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:20.578438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0956c-8cbc-4beb-9f01-41a8c906e72a None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:20.578710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0956c-8cbc-4beb-9f01-41a8c906e72a None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:20.578972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0956c-8cbc-4beb-9f01-41a8c906e72a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:20.586035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0956c-8cbc-4beb-9f01-41a8c906e72a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:20.587440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0956c-8cbc-4beb-9f01-41a8c906e72a None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-b2e0956c-8cbc-4beb-9f01-41a8c906e72a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:33:20.598752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:20.599776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:20.601007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:33:20.601007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:33:20.601007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:33:20.601007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:33:20.601007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:33:20.601007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:33:20.601007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:20.601007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:20.601007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 644/1271] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:33:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:20.605630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23428336-4872-443a-bb66-19cce1f2f4a3 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:20.662669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:20.702787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23428336-4872-443a-bb66-19cce1f2f4a3 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:20.735169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d97cd4-19d6-4814-92ae-8a08425f4bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:20.735755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97cd4-19d6-4814-92ae-8a08425f4bde None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:20.736028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97cd4-19d6-4814-92ae-8a08425f4bde None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:20.736293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d97cd4-19d6-4814-92ae-8a08425f4bde None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:20.736995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d97cd4-19d6-4814-92ae-8a08425f4bde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:20.737292 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d97cd4-19d6-4814-92ae-8a08425f4bde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:20.738999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d97cd4-19d6-4814-92ae-8a08425f4bde None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:20.750310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 628/1272] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:33:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:20.756340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0956c-8cbc-4beb-9f01-41a8c906e72a None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:20.846155 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:20.855599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0956c-8cbc-4beb-9f01-41a8c906e72a None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:20.887939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37720c9a-fded-4798-a406-374384f9daa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:20.888579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37720c9a-fded-4798-a406-374384f9daa9 None admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:20.888882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37720c9a-fded-4798-a406-374384f9daa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:20.889153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37720c9a-fded-4798-a406-374384f9daa9 None admin] PATH_INFO: `/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/1d7e1b7dfa32448a88e0c90f10cdb2e9/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:20.891026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37720c9a-fded-4798-a406-374384f9daa9 None admin] RBAC: Authorizing `identity:create_grant(project_id=fbb4b236264d4062a720496ccb7cbbfe, user_id=1d7e1b7dfa32448a88e0c90f10cdb2e9, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:20.892435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37720c9a-fded-4798-a406-374384f9daa9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:20.905223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 645/1273] 10.222.0.39 () {58 vars in 1439 bytes} [Tue Sep 6 22:33:20 2022] PUT /identity/v3/projects/fbb4b236264d4062a720496ccb7cbbfe/users/1d7e1b7dfa32448a88e0c90f10cdb2e9/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:33:23.029348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a18c0-f3c8-418e-b848-e1a6105c5cea None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:23.029903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a18c0-f3c8-418e-b848-e1a6105c5cea None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:23.029903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a18c0-f3c8-418e-b848-e1a6105c5cea None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:23.031402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 629/1274] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:33:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:33:23.037266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94879d7b-a0f9-46c6-9f09-f8a33001b7db None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:23.037713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94879d7b-a0f9-46c6-9f09-f8a33001b7db None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:23.037842 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94879d7b-a0f9-46c6-9f09-f8a33001b7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:23.057130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-94879d7b-a0f9-46c6-9f09-f8a33001b7db None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:33:23.058308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-94879d7b-a0f9-46c6-9f09-f8a33001b7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:33:23.125361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 646/1275] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:33:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:33:23.391283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a18c0-f3c8-418e-b848-e1a6105c5cea None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:23.490375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a18c0-f3c8-418e-b848-e1a6105c5cea None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:23.507253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:23.524192 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538ab6c-850f-48df-84a5-d03e53d4000a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:23.524826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538ab6c-850f-48df-84a5-d03e53d4000a None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:23.525102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538ab6c-850f-48df-84a5-d03e53d4000a None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:23.525365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5538ab6c-850f-48df-84a5-d03e53d4000a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:23.535025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5538ab6c-850f-48df-84a5-d03e53d4000a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-5538ab6c-850f-48df-84a5-d03e53d4000a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 06 22:33:23.539729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:23.540859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:23.541884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 630/1276] 10.222.0.39 () {56 vars in 1096 bytes} [Tue Sep 6 22:33:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:23.546756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94879d7b-a0f9-46c6-9f09-f8a33001b7db None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:23.626798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:23.646011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94879d7b-a0f9-46c6-9f09-f8a33001b7db None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:23.678627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22eae9-d9a7-4f82-84c0-95a29dd8826b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:23.679223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22eae9-d9a7-4f82-84c0-95a29dd8826b None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:23.679498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22eae9-d9a7-4f82-84c0-95a29dd8826b None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:23.679766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad22eae9-d9a7-4f82-84c0-95a29dd8826b None admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:23.680416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22eae9-d9a7-4f82-84c0-95a29dd8826b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:23.680714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22eae9-d9a7-4f82-84c0-95a29dd8826b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:23.682096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad22eae9-d9a7-4f82-84c0-95a29dd8826b None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:23.689798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 647/1277] 10.222.0.39 () {56 vars in 1099 bytes} [Tue Sep 6 22:33:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:23.695693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538ab6c-850f-48df-84a5-d03e53d4000a None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:23.793802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5538ab6c-850f-48df-84a5-d03e53d4000a None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:23.827350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00275749-2ab2-45c9-bb8f-131677fc2460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:23.827965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00275749-2ab2-45c9-bb8f-131677fc2460 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:23.828242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00275749-2ab2-45c9-bb8f-131677fc2460 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:23.828511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00275749-2ab2-45c9-bb8f-131677fc2460 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:23.838275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00275749-2ab2-45c9-bb8f-131677fc2460 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:23.839704 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00275749-2ab2-45c9-bb8f-131677fc2460 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-00275749-2ab2-45c9-bb8f-131677fc2460 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:23.853977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:23.855126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:23.856365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:23.856365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:23.856365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:23.856365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 631/1278] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:23.860782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22eae9-d9a7-4f82-84c0-95a29dd8826b None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:23.958746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad22eae9-d9a7-4f82-84c0-95a29dd8826b None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:23.993470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8746-4556-433f-9024-b1f52a631c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:23.994076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8746-4556-433f-9024-b1f52a631c14 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:23.994353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8746-4556-433f-9024-b1f52a631c14 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:23.994623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239c8746-4556-433f-9024-b1f52a631c14 None admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:23.995257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c8746-4556-433f-9024-b1f52a631c14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:23.995555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c8746-4556-433f-9024-b1f52a631c14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:23.996936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239c8746-4556-433f-9024-b1f52a631c14 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:24.008501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 648/1279] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:24.014599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00275749-2ab2-45c9-bb8f-131677fc2460 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:24.044571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:24.111497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00275749-2ab2-45c9-bb8f-131677fc2460 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:24.144780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75dd34-ad40-411a-936e-4eb86db5ba8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:24.145414 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75dd34-ad40-411a-936e-4eb86db5ba8c None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:24.145687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75dd34-ad40-411a-936e-4eb86db5ba8c None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:24.145955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee75dd34-ad40-411a-936e-4eb86db5ba8c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:24.153085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75dd34-ad40-411a-936e-4eb86db5ba8c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:24.154504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee75dd34-ad40-411a-936e-4eb86db5ba8c None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-ee75dd34-ad40-411a-936e-4eb86db5ba8c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 06 22:33:24.166065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:24.167187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:24.168382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:24.168382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:24.168382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 06 22:33:24.168382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 06 22:33:24.168382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 06 22:33:24.168382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 06 22:33:24.168382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 06 22:33:24.168382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 06 22:33:24.168382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 06 22:33:24.168382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:24.168382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 632/1280] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:24.172889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8746-4556-433f-9024-b1f52a631c14 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:24.177325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:24.269872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239c8746-4556-433f-9024-b1f52a631c14 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:24.303713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a913675-7858-4d90-ad9c-89bd3e05a02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:24.304307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913675-7858-4d90-ad9c-89bd3e05a02d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:24.304579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913675-7858-4d90-ad9c-89bd3e05a02d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:24.304874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a913675-7858-4d90-ad9c-89bd3e05a02d None admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:24.305587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a913675-7858-4d90-ad9c-89bd3e05a02d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:24.305885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a913675-7858-4d90-ad9c-89bd3e05a02d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:24.307296 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a913675-7858-4d90-ad9c-89bd3e05a02d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:24.324119 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 649/1281] 10.222.0.39 () {56 vars in 1143 bytes} [Tue Sep 6 22:33:24 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:24.329860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75dd34-ad40-411a-936e-4eb86db5ba8c None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:24.419800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:24.429354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee75dd34-ad40-411a-936e-4eb86db5ba8c None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:24.462085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e217487-2776-43af-9583-10edf96a8740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:24.462681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e217487-2776-43af-9583-10edf96a8740 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:24.462958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e217487-2776-43af-9583-10edf96a8740 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:24.463225 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e217487-2776-43af-9583-10edf96a8740 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:24.528441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e217487-2776-43af-9583-10edf96a8740 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:24.529847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e217487-2776-43af-9583-10edf96a8740 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:24.625246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:24.635688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 633/1282] 10.222.0.39 () {58 vars in 1282 bytes} [Tue Sep 6 22:33:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 06 22:33:24.641723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a913675-7858-4d90-ad9c-89bd3e05a02d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:24.701861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:24.739561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a913675-7858-4d90-ad9c-89bd3e05a02d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:24.772658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460150ff-7051-4316-a9b8-ed1848bb53e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:24.773289 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460150ff-7051-4316-a9b8-ed1848bb53e1 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:24.773561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460150ff-7051-4316-a9b8-ed1848bb53e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:24.773825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460150ff-7051-4316-a9b8-ed1848bb53e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:24.783337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460150ff-7051-4316-a9b8-ed1848bb53e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:24.784736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460150ff-7051-4316-a9b8-ed1848bb53e1 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-460150ff-7051-4316-a9b8-ed1848bb53e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 06 22:33:24.798551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 06 22:33:24.801100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 06 22:33:24.801100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 06 22:33:24.801100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:24.801100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 650/1283] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:33:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:24.805558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e217487-2776-43af-9583-10edf96a8740 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:24.901459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e217487-2776-43af-9583-10edf96a8740 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:24.934219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5454ef9-fb0e-4ebe-ac76-0fef21423ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:24.934803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5454ef9-fb0e-4ebe-ac76-0fef21423ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:24.935075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5454ef9-fb0e-4ebe-ac76-0fef21423ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:24.935341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5454ef9-fb0e-4ebe-ac76-0fef21423ab5 None admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:24.935964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5454ef9-fb0e-4ebe-ac76-0fef21423ab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:24.936255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5454ef9-fb0e-4ebe-ac76-0fef21423ab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:24.937623 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5454ef9-fb0e-4ebe-ac76-0fef21423ab5 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:24.948784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 634/1284] 10.222.0.39 () {56 vars in 1109 bytes} [Tue Sep 6 22:33:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:24.954476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460150ff-7051-4316-a9b8-ed1848bb53e1 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:25.051504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460150ff-7051-4316-a9b8-ed1848bb53e1 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:25.064265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:25.083939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09621940-cc3e-46a0-b626-04034b1d1de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:25.084535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621940-cc3e-46a0-b626-04034b1d1de0 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:25.084836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621940-cc3e-46a0-b626-04034b1d1de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:25.085102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621940-cc3e-46a0-b626-04034b1d1de0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:25.091348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09621940-cc3e-46a0-b626-04034b1d1de0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:25.092743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09621940-cc3e-46a0-b626-04034b1d1de0 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application [None req-09621940-cc3e-46a0-b626-04034b1d1de0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 06 22:33:25.103499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:25.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:25.105962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 06 22:33:25.105962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 06 22:33:25.105962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 06 22:33:25.105962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 06 22:33:25.105962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 06 22:33:25.105962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 06 22:33:25.105962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 06 22:33:25.105962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: ERROR keystone.server.flask.application Sep 06 22:33:25.105962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 651/1285] 10.222.0.39 () {56 vars in 1111 bytes} [Tue Sep 6 22:33:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:25.110396 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5454ef9-fb0e-4ebe-ac76-0fef21423ab5 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:25.160030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:25.209788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5454ef9-fb0e-4ebe-ac76-0fef21423ab5 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:25.242220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f3544-18d6-453f-9840-3c024a0b9721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:25.242814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3544-18d6-453f-9840-3c024a0b9721 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:25.243092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3544-18d6-453f-9840-3c024a0b9721 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:25.243363 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3544-18d6-453f-9840-3c024a0b9721 None admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:25.244045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3544-18d6-453f-9840-3c024a0b9721 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:25.244348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3544-18d6-453f-9840-3c024a0b9721 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:25.245796 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3544-18d6-453f-9840-3c024a0b9721 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:25.257894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 635/1286] 10.222.0.39 () {56 vars in 1148 bytes} [Tue Sep 6 22:33:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:25.264192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09621940-cc3e-46a0-b626-04034b1d1de0 None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:25.359796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09621940-cc3e-46a0-b626-04034b1d1de0 None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:25.392730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1eda69-09c8-473d-a45e-985d1514f34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:25.393337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eda69-09c8-473d-a45e-985d1514f34e None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:25.393611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eda69-09c8-473d-a45e-985d1514f34e None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:25.393947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1eda69-09c8-473d-a45e-985d1514f34e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:25.394670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1eda69-09c8-473d-a45e-985d1514f34e None admin] RBAC: Adding query filter params (role.id=7a917eb93a9d4f8187269ed155e8e387, scope.project.id=fbb4b236264d4062a720496ccb7cbbfe, user.id=1d7e1b7dfa32448a88e0c90f10cdb2e9) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:25.394973 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1eda69-09c8-473d-a45e-985d1514f34e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:25.396391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1eda69-09c8-473d-a45e-985d1514f34e None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:25.401927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 652/1287] 10.222.0.39 () {56 vars in 1374 bytes} [Tue Sep 6 22:33:25 2022] GET /identity/v3/role_assignments?user.id=1d7e1b7dfa32448a88e0c90f10cdb2e9&scope.project.id=fbb4b236264d4062a720496ccb7cbbfe&role.id=7a917eb93a9d4f8187269ed155e8e387 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:27.546790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a94b9-ea1f-46f9-9502-0518257bd1aa None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:27.547360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a94b9-ea1f-46f9-9502-0518257bd1aa None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:27.547360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a94b9-ea1f-46f9-9502-0518257bd1aa None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:27.548682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 636/1288] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:33:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:33:27.554570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944de969-aa35-4297-8ca9-0a1144443f29 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:27.554983 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944de969-aa35-4297-8ca9-0a1144443f29 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:27.555101 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944de969-aa35-4297-8ca9-0a1144443f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:27.573766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-944de969-aa35-4297-8ca9-0a1144443f29 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:33:27.574953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-944de969-aa35-4297-8ca9-0a1144443f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:33:27.604185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:27.641374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 653/1289] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:33:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:33:27.910267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a94b9-ea1f-46f9-9502-0518257bd1aa None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:28.023680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a94b9-ea1f-46f9-9502-0518257bd1aa None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:28.035346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:28.058532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b200d56-d91c-4eb5-85f3-2072eab77008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:28.059145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b200d56-d91c-4eb5-85f3-2072eab77008 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:28.059416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b200d56-d91c-4eb5-85f3-2072eab77008 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:28.059688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b200d56-d91c-4eb5-85f3-2072eab77008 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:28.060254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b200d56-d91c-4eb5-85f3-2072eab77008 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:28.061641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b200d56-d91c-4eb5-85f3-2072eab77008 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-3b200d56-d91c-4eb5-85f3-2072eab77008 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:28.070742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:28.071980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:28.073087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 637/1290] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Sep 6 22:33:27 2022] GET /identity/v3/services/placement => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:28.077784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944de969-aa35-4297-8ca9-0a1144443f29 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:28.171456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944de969-aa35-4297-8ca9-0a1144443f29 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:28.204065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac210fc5-8d5b-498b-9e7a-62b484b952a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:28.204649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac210fc5-8d5b-498b-9e7a-62b484b952a9 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:28.204945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac210fc5-8d5b-498b-9e7a-62b484b952a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:28.205210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac210fc5-8d5b-498b-9e7a-62b484b952a9 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:28.205815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac210fc5-8d5b-498b-9e7a-62b484b952a9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:28.206106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac210fc5-8d5b-498b-9e7a-62b484b952a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:28.207961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac210fc5-8d5b-498b-9e7a-62b484b952a9 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:28.212527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 654/1291] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Sep 6 22:33:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:28.218844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b200d56-d91c-4eb5-85f3-2072eab77008 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:28.325912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b200d56-d91c-4eb5-85f3-2072eab77008 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:28.357684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c2c30b-ca55-45cb-8694-a4332e98842d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:28.358282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2c30b-ca55-45cb-8694-a4332e98842d None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:28.358557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2c30b-ca55-45cb-8694-a4332e98842d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:28.358826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c2c30b-ca55-45cb-8694-a4332e98842d None admin] PATH_INFO: `/v3/services` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:28.359445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2c30b-ca55-45cb-8694-a4332e98842d None admin] RBAC: Adding query filter params (type=placement) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:28.359750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2c30b-ca55-45cb-8694-a4332e98842d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:28.361139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c2c30b-ca55-45cb-8694-a4332e98842d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:28.366051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 638/1292] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Sep 6 22:33:28 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:33:30.519355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d0745-64f2-45ed-8ca8-7f73636a59df None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:30.519996 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d0745-64f2-45ed-8ca8-7f73636a59df None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:30.519996 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d0745-64f2-45ed-8ca8-7f73636a59df None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:30.521250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 655/1293] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:33:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:33:30.526478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd75a0-d219-4eb4-bb5b-2e11d51894e8 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:30.526903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd75a0-d219-4eb4-bb5b-2e11d51894e8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:30.527026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cd75a0-d219-4eb4-bb5b-2e11d51894e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:30.544942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-44cd75a0-d219-4eb4-bb5b-2e11d51894e8 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:33:30.546142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-44cd75a0-d219-4eb4-bb5b-2e11d51894e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:33:30.575918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:30.614480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 639/1294] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:33:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:33:30.882665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d0745-64f2-45ed-8ca8-7f73636a59df None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:30.914629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:30.985760 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d0745-64f2-45ed-8ca8-7f73636a59df None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:31.019395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ed711a-41ec-4257-959b-c6b73234ee15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:31.020137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed711a-41ec-4257-959b-c6b73234ee15 None admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:31.020410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed711a-41ec-4257-959b-c6b73234ee15 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:31.020674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ed711a-41ec-4257-959b-c6b73234ee15 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:31.021337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed711a-41ec-4257-959b-c6b73234ee15 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:31.022687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ed711a-41ec-4257-959b-c6b73234ee15 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:31.037263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 656/1295] 10.222.0.39 () {60 vars in 1141 bytes} [Tue Sep 6 22:33:30 2022] POST /identity/v3/services => generated 246 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:33:33.176362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd376d7-141e-4ed4-ac14-8b7e799b8d5d None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:33.176958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd376d7-141e-4ed4-ac14-8b7e799b8d5d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:33.177070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd376d7-141e-4ed4-ac14-8b7e799b8d5d None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:33.178380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 640/1296] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:33:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:33:33.183480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c3011-cf4a-455c-a84c-55d104efd5a6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:33.183875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c3011-cf4a-455c-a84c-55d104efd5a6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:33.184028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c3011-cf4a-455c-a84c-55d104efd5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:33.202236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-028c3011-cf4a-455c-a84c-55d104efd5a6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:33:33.203416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-028c3011-cf4a-455c-a84c-55d104efd5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:33:33.274236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 657/1297] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:33:33 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:33:33.540698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd376d7-141e-4ed4-ac14-8b7e799b8d5d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:33.641727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd376d7-141e-4ed4-ac14-8b7e799b8d5d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:33.674358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9c1e3-789d-4ee5-bdf1-f0c449896a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:33.674968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9c1e3-789d-4ee5-bdf1-f0c449896a63 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:33.675770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9c1e3-789d-4ee5-bdf1-f0c449896a63 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:33.676073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9c1e3-789d-4ee5-bdf1-f0c449896a63 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:33.676605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9c1e3-789d-4ee5-bdf1-f0c449896a63 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:33.677989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9c1e3-789d-4ee5-bdf1-f0c449896a63 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-12c9c1e3-789d-4ee5-bdf1-f0c449896a63 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:33.686455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:33.687764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:33.689102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 641/1298] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Sep 6 22:33:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:33.693635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c3011-cf4a-455c-a84c-55d104efd5a6 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:33.711126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:33.791095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028c3011-cf4a-455c-a84c-55d104efd5a6 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:33.823890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39ef99-c9f8-4a60-9116-db677b9d845d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:33.824479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39ef99-c9f8-4a60-9116-db677b9d845d None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:33.824747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39ef99-c9f8-4a60-9116-db677b9d845d None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:33.825036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee39ef99-c9f8-4a60-9116-db677b9d845d None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:33.825642 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39ef99-c9f8-4a60-9116-db677b9d845d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:33.825934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39ef99-c9f8-4a60-9116-db677b9d845d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:33.827262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee39ef99-c9f8-4a60-9116-db677b9d845d None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:33.832127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 658/1299] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Sep 6 22:33:33 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:33.838070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9c1e3-789d-4ee5-bdf1-f0c449896a63 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:33.869621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:33.937610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9c1e3-789d-4ee5-bdf1-f0c449896a63 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:33.974475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db388071-5a09-4765-9a77-8fdb8a8d1205 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:33.975337 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db388071-5a09-4765-9a77-8fdb8a8d1205 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:33.975626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db388071-5a09-4765-9a77-8fdb8a8d1205 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:33.976005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db388071-5a09-4765-9a77-8fdb8a8d1205 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:33.976904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db388071-5a09-4765-9a77-8fdb8a8d1205 None admin] RBAC: Adding query filter params (interface=public, service_id=15c8070082204bd29160bff979377e8b, region_id=RegionOne) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:33.977338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db388071-5a09-4765-9a77-8fdb8a8d1205 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:33.979094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db388071-5a09-4765-9a77-8fdb8a8d1205 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:33.984721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 642/1300] 10.222.0.39 () {56 vars in 1251 bytes} [Tue Sep 6 22:33:33 2022] GET /identity/v3/endpoints?service_id=15c8070082204bd29160bff979377e8b&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:33.992865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39ef99-c9f8-4a60-9116-db677b9d845d None admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:34.079735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:34.098381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee39ef99-c9f8-4a60-9116-db677b9d845d None admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:34.130502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b840d84-69b1-4a63-8840-45dfac3a4144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:34.131094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b840d84-69b1-4a63-8840-45dfac3a4144 None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:34.131371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b840d84-69b1-4a63-8840-45dfac3a4144 None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:34.131646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b840d84-69b1-4a63-8840-45dfac3a4144 None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:34.132234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b840d84-69b1-4a63-8840-45dfac3a4144 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:34.133606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b840d84-69b1-4a63-8840-45dfac3a4144 None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:34.140380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 659/1301] 10.222.0.39 () {56 vars in 1087 bytes} [Tue Sep 6 22:33:33 2022] GET /identity/v3/services => generated 2191 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:33:36.257884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ab1df-eeaf-4d69-9dc2-8798f07832b5 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:36.258439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ab1df-eeaf-4d69-9dc2-8798f07832b5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:36.258439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0ab1df-eeaf-4d69-9dc2-8798f07832b5 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:36.259765 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 643/1302] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:33:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:33:36.265186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d289f-8e0a-4a19-9244-3eaf74ca3615 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:36.265572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d289f-8e0a-4a19-9244-3eaf74ca3615 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:36.265729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01d289f-8e0a-4a19-9244-3eaf74ca3615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:36.283769 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c01d289f-8e0a-4a19-9244-3eaf74ca3615 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:33:36.285068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c01d289f-8e0a-4a19-9244-3eaf74ca3615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:33:36.349704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 660/1303] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:33:36 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 06 22:33:36.617594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ab1df-eeaf-4d69-9dc2-8798f07832b5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:36.713009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:36.719889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0ab1df-eeaf-4d69-9dc2-8798f07832b5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:36.752990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584be01a-47e9-4e2d-a798-d6785e78b2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:36.753664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584be01a-47e9-4e2d-a798-d6785e78b2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:36.753900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584be01a-47e9-4e2d-a798-d6785e78b2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:36.754178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584be01a-47e9-4e2d-a798-d6785e78b2e4 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:36.754800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584be01a-47e9-4e2d-a798-d6785e78b2e4 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:36.756153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584be01a-47e9-4e2d-a798-d6785e78b2e4 None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application [None req-584be01a-47e9-4e2d-a798-d6785e78b2e4 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 06 22:33:36.765133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 06 22:33:36.766295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.get_or_create( Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application with Lock( Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self._enter() Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application return self.creator() Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application created_value = creator( Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: ERROR keystone.server.flask.application Sep 06 22:33:36.767182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 644/1304] 10.222.0.39 () {56 vars in 1117 bytes} [Tue Sep 6 22:33:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 06 22:33:36.772249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d289f-8e0a-4a19-9244-3eaf74ca3615 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:36.845956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:36.869897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01d289f-8e0a-4a19-9244-3eaf74ca3615 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:36.902860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cb36f4-e684-4f4a-83ea-e282b1a2bcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:36.903451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb36f4-e684-4f4a-83ea-e282b1a2bcad None admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:36.903727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb36f4-e684-4f4a-83ea-e282b1a2bcad None admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:36.903997 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cb36f4-e684-4f4a-83ea-e282b1a2bcad None admin] PATH_INFO: `/v3/services` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:36.904611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb36f4-e684-4f4a-83ea-e282b1a2bcad None admin] RBAC: Adding query filter params (name=placement) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:33:36.904941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb36f4-e684-4f4a-83ea-e282b1a2bcad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:36.906285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cb36f4-e684-4f4a-83ea-e282b1a2bcad None admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:36.911915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 661/1305] 10.222.0.39 () {56 vars in 1116 bytes} [Tue Sep 6 22:33:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:33:36.917704 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584be01a-47e9-4e2d-a798-d6785e78b2e4 None admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:33:37.016874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584be01a-47e9-4e2d-a798-d6785e78b2e4 None admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:33:37.050721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d8e2-6a41-415a-b157-2d9f7da2379d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:33:37.051311 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d8e2-6a41-415a-b157-2d9f7da2379d None admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:33:37.051583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d8e2-6a41-415a-b157-2d9f7da2379d None admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:33:37.051846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d8e2-6a41-415a-b157-2d9f7da2379d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:33:37.052498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372d8e2-6a41-415a-b157-2d9f7da2379d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:33:37.053872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372d8e2-6a41-415a-b157-2d9f7da2379d None admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:33:37.072172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 645/1306] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:33:36 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:33:46.908396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=30 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:33:47.068024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:41.585655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63698d-be17-4b3a-a3ea-521f3ec90493 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:41.587064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63698d-be17-4b3a-a3ea-521f3ec90493 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:41.587633 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e63698d-be17-4b3a-a3ea-521f3ec90493 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:41.594587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 662/1307] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:34:41 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:34:41.596090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdfd2c-c9cc-4360-b725-4675dba5fb82 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:41.596522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdfd2c-c9cc-4360-b725-4675dba5fb82 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:41.596522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4cdfd2c-c9cc-4360-b725-4675dba5fb82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:41.691082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e4cdfd2c-c9cc-4360-b725-4675dba5fb82 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:34:41.693005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e4cdfd2c-c9cc-4360-b725-4675dba5fb82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:41.812801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 646/1308] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:34:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:34:42.106229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab58906-621b-4ca9-91ee-510e671876b0 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:42.106587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab58906-621b-4ca9-91ee-510e671876b0 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:42.107103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab58906-621b-4ca9-91ee-510e671876b0 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:42.108519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 663/1309] 10.222.0.39 () {54 vars in 965 bytes} [Tue Sep 6 22:34:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:34:42.114545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d8eaa-cad8-45d1-9ce4-acda7424c76c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:42.114905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d8eaa-cad8-45d1-9ce4-acda7424c76c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:42.115254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4d8eaa-cad8-45d1-9ce4-acda7424c76c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:42.204364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1c4d8eaa-cad8-45d1-9ce4-acda7424c76c None None] MFA Rules not processed for user `2658435f74a947f39e207465ebff22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:34:42.205748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1c4d8eaa-cad8-45d1-9ce4-acda7424c76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:42.230729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:42.299922 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 647/1310] 10.222.0.39 () {58 vars in 1062 bytes} [Tue Sep 6 22:34:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:34:42.308598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab58906-621b-4ca9-91ee-510e671876b0 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:34:42.310412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5ab58906-621b-4ca9-91ee-510e671876b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:42.433196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab58906-621b-4ca9-91ee-510e671876b0 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:34:42.510421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6920aec-b8b4-4005-a276-d7bc15be5cc5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:34:42.511026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6920aec-b8b4-4005-a276-d7bc15be5cc5 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:42.511300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6920aec-b8b4-4005-a276-d7bc15be5cc5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:42.511566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6920aec-b8b4-4005-a276-d7bc15be5cc5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:42.513169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c6920aec-b8b4-4005-a276-d7bc15be5cc5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:42.577876 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:42.633121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6920aec-b8b4-4005-a276-d7bc15be5cc5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:34:42.634542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6920aec-b8b4-4005-a276-d7bc15be5cc5 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:34:42.655204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 664/1311] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:34:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:34:45.153300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6a1ca-b284-4abb-bf62-1879b9637921 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:45.153852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6a1ca-b284-4abb-bf62-1879b9637921 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:45.153852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f6a1ca-b284-4abb-bf62-1879b9637921 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:45.155200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 648/1312] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:34:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:34:45.160254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04add865-019b-44c9-82dc-056786dcda1d None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:45.160685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04add865-019b-44c9-82dc-056786dcda1d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:45.160810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04add865-019b-44c9-82dc-056786dcda1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:45.240707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-04add865-019b-44c9-82dc-056786dcda1d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:34:45.241942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-04add865-019b-44c9-82dc-056786dcda1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:45.340669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 665/1313] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:34:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:34:45.641422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6a1ca-b284-4abb-bf62-1879b9637921 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:34:45.642962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d8f6a1ca-b284-4abb-bf62-1879b9637921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:45.747985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f6a1ca-b284-4abb-bf62-1879b9637921 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:34:45.833490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:45.836266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd9e13a-19fb-4c5b-a332-dd3b1567eaf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:34:45.836911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9e13a-19fb-4c5b-a332-dd3b1567eaf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:45.837377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9e13a-19fb-4c5b-a332-dd3b1567eaf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:45.837798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd9e13a-19fb-4c5b-a332-dd3b1567eaf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:45.839586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4dd9e13a-19fb-4c5b-a332-dd3b1567eaf6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:45.965399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9e13a-19fb-4c5b-a332-dd3b1567eaf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:34:45.966891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd9e13a-19fb-4c5b-a332-dd3b1567eaf6 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:34:45.988891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 649/1314] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:34:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:34:48.513661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b92575-d78e-4482-a96e-cd60c800c533 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:48.514201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b92575-d78e-4482-a96e-cd60c800c533 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:48.514201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b92575-d78e-4482-a96e-cd60c800c533 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:48.515836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 666/1315] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:34:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:34:48.521207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c194fd-1f7a-4ae6-b5a2-73d297108542 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:48.521597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c194fd-1f7a-4ae6-b5a2-73d297108542 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:48.521750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c194fd-1f7a-4ae6-b5a2-73d297108542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:48.587740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f3c194fd-1f7a-4ae6-b5a2-73d297108542 None None] MFA Rules not processed for user `0cac9a0d6d824acfbc13df63c8e5dff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:34:48.589019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f3c194fd-1f7a-4ae6-b5a2-73d297108542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:48.640852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:48.685904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 650/1316] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:34:48 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:34:48.974176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b92575-d78e-4482-a96e-cd60c800c533 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:34:48.975718 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e5b92575-d78e-4482-a96e-cd60c800c533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:48.985420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:49.078260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b92575-d78e-4482-a96e-cd60c800c533 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:34:49.152010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4bfd47-ad20-4c8b-9dd2-342297922a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:34:49.152541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bfd47-ad20-4c8b-9dd2-342297922a8d service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:49.152853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bfd47-ad20-4c8b-9dd2-342297922a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:49.153124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4bfd47-ad20-4c8b-9dd2-342297922a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:49.154731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2b4bfd47-ad20-4c8b-9dd2-342297922a8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:49.215163 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:49.261007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bfd47-ad20-4c8b-9dd2-342297922a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:34:49.262419 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4bfd47-ad20-4c8b-9dd2-342297922a8d service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:34:49.281284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 667/1317] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:34:48 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:34:51.955214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d44d7-5bd4-4f1f-b1af-78246518163d None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:51.955783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d44d7-5bd4-4f1f-b1af-78246518163d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:51.955783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1d44d7-5bd4-4f1f-b1af-78246518163d None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:51.957120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 651/1318] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:34:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:34:51.962858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009b81c-0967-4c95-81ef-3b3b613338ba None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:51.963251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009b81c-0967-4c95-81ef-3b3b613338ba None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:51.963644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009b81c-0967-4c95-81ef-3b3b613338ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:52.031210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-4009b81c-0967-4c95-81ef-3b3b613338ba None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:34:52.032435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4009b81c-0967-4c95-81ef-3b3b613338ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:52.129186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 668/1319] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:34:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:34:52.411113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d44d7-5bd4-4f1f-b1af-78246518163d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:34:52.412663 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5e1d44d7-5bd4-4f1f-b1af-78246518163d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:52.514278 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1d44d7-5bd4-4f1f-b1af-78246518163d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:34:52.587161 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02532419-aa62-4c16-b7ac-49c932c64eeb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:34:52.587761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02532419-aa62-4c16-b7ac-49c932c64eeb service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:52.588034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02532419-aa62-4c16-b7ac-49c932c64eeb service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:52.588301 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02532419-aa62-4c16-b7ac-49c932c64eeb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:52.589942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-02532419-aa62-4c16-b7ac-49c932c64eeb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:52.597277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:52.698158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02532419-aa62-4c16-b7ac-49c932c64eeb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:34:52.699529 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02532419-aa62-4c16-b7ac-49c932c64eeb service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:34:52.717199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 652/1320] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:34:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:34:55.493896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2379e-1b07-4c55-88d5-9dda3461df2e None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:55.494431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2379e-1b07-4c55-88d5-9dda3461df2e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:55.494431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab2379e-1b07-4c55-88d5-9dda3461df2e None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:55.495822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 669/1321] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:34:55.501078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e206-f762-4508-9f99-90830197b716 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:55.501507 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e206-f762-4508-9f99-90830197b716 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:55.501601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483e206-f762-4508-9f99-90830197b716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:55.570399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7483e206-f762-4508-9f99-90830197b716 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:34:55.571664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7483e206-f762-4508-9f99-90830197b716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:55.634390 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:55.678077 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 653/1322] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:34:55.975121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721be0-a6b1-47de-9a07-59378f8f0f5b None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:55.975364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721be0-a6b1-47de-9a07-59378f8f0f5b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:55.975582 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721be0-a6b1-47de-9a07-59378f8f0f5b None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:55.976939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 670/1323] 10.222.0.39 () {54 vars in 965 bytes} [Tue Sep 6 22:34:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:34:55.982938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4746b5-b207-4eae-a1f7-d3c50ba31acd None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:55.983224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4746b5-b207-4eae-a1f7-d3c50ba31acd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:55.983496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4746b5-b207-4eae-a1f7-d3c50ba31acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:56.059480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8f4746b5-b207-4eae-a1f7-d3c50ba31acd None None] MFA Rules not processed for user `2658435f74a947f39e207465ebff22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:34:56.060689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8f4746b5-b207-4eae-a1f7-d3c50ba31acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:56.135370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 654/1324] 10.222.0.39 () {58 vars in 1062 bytes} [Tue Sep 6 22:34:55 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:34:56.144562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1721be0-a6b1-47de-9a07-59378f8f0f5b None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:34:56.146655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c1721be0-a6b1-47de-9a07-59378f8f0f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:56.173527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:56.266156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1721be0-a6b1-47de-9a07-59378f8f0f5b None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:34:56.343252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15474980-f58f-4d4f-a4cb-260ddbf69a72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:34:56.343852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15474980-f58f-4d4f-a4cb-260ddbf69a72 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:56.344125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15474980-f58f-4d4f-a4cb-260ddbf69a72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:56.344392 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15474980-f58f-4d4f-a4cb-260ddbf69a72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:56.346049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-15474980-f58f-4d4f-a4cb-260ddbf69a72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:56.419344 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:56.456922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15474980-f58f-4d4f-a4cb-260ddbf69a72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:34:56.458335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15474980-f58f-4d4f-a4cb-260ddbf69a72 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:34:56.477257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 671/1325] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:34:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:34:58.907725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91a443-e1f5-4a40-89ec-e010f363003e None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:58.908240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91a443-e1f5-4a40-89ec-e010f363003e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:58.908240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b91a443-e1f5-4a40-89ec-e010f363003e None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:58.909624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 655/1326] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:34:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:34:58.915087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb177ea6-77db-432e-9ff6-ff26987d3a60 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:58.915493 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb177ea6-77db-432e-9ff6-ff26987d3a60 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:58.915648 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb177ea6-77db-432e-9ff6-ff26987d3a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:58.982642 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-bb177ea6-77db-432e-9ff6-ff26987d3a60 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:34:58.983760 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-bb177ea6-77db-432e-9ff6-ff26987d3a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:59.078650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 672/1327] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:34:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:34:59.362739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91a443-e1f5-4a40-89ec-e010f363003e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:34:59.364412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1b91a443-e1f5-4a40-89ec-e010f363003e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:59.426228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:59.470928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b91a443-e1f5-4a40-89ec-e010f363003e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:34:59.544649 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1e355b-8b36-42c7-870f-164509250be7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:34:59.545274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e355b-8b36-42c7-870f-164509250be7 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:34:59.545546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e355b-8b36-42c7-870f-164509250be7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:34:59.545808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e355b-8b36-42c7-870f-164509250be7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:34:59.547384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8c1e355b-8b36-42c7-870f-164509250be7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:34:59.659760 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:34:59.662568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e355b-8b36-42c7-870f-164509250be7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:34:59.664346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1e355b-8b36-42c7-870f-164509250be7 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:34:59.682900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 656/1328] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:34:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:01.947617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832525a-527c-4d3c-b34b-c927218bad21 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:01.948121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832525a-527c-4d3c-b34b-c927218bad21 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:01.948121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832525a-527c-4d3c-b34b-c927218bad21 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:01.949461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 673/1329] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:01.954916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca322-5cd2-4200-bace-477b737bd985 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:01.955632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca322-5cd2-4200-bace-477b737bd985 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:01.955866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479ca322-5cd2-4200-bace-477b737bd985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:02.029365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-479ca322-5cd2-4200-bace-477b737bd985 None None] MFA Rules not processed for user `0cac9a0d6d824acfbc13df63c8e5dff8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:02.030571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-479ca322-5cd2-4200-bace-477b737bd985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:02.131803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 657/1330] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:01 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:02.416809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832525a-527c-4d3c-b34b-c927218bad21 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:02.418366 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2832525a-527c-4d3c-b34b-c927218bad21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:02.449821 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:02.525038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832525a-527c-4d3c-b34b-c927218bad21 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:02.603450 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e74b0ef-0d3d-41d0-b20f-c0f614c8df8c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:02.604052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74b0ef-0d3d-41d0-b20f-c0f614c8df8c service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:02.604329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74b0ef-0d3d-41d0-b20f-c0f614c8df8c service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:02.604598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e74b0ef-0d3d-41d0-b20f-c0f614c8df8c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:02.606241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5e74b0ef-0d3d-41d0-b20f-c0f614c8df8c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:02.688334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:02.720837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74b0ef-0d3d-41d0-b20f-c0f614c8df8c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:02.722271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e74b0ef-0d3d-41d0-b20f-c0f614c8df8c service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:02.743423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 674/1331] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:02 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:05.147810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47499cd6-98ef-4948-9ab6-305d59d1222b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:05.148326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47499cd6-98ef-4948-9ab6-305d59d1222b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:05.148326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47499cd6-98ef-4948-9ab6-305d59d1222b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:05.149660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 658/1332] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:05.154981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818d207-4cd3-40d5-acf1-992443947155 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:05.155365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818d207-4cd3-40d5-acf1-992443947155 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:05.155529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7818d207-4cd3-40d5-acf1-992443947155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:05.221774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7818d207-4cd3-40d5-acf1-992443947155 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:05.222971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7818d207-4cd3-40d5-acf1-992443947155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:05.319194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 675/1333] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:05.601674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47499cd6-98ef-4948-9ab6-305d59d1222b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:05.603238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-47499cd6-98ef-4948-9ab6-305d59d1222b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:05.671586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:05.705964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47499cd6-98ef-4948-9ab6-305d59d1222b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:05.778451 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15aa4371-fe60-4375-9bcc-58d9bc90de22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:05.779048 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa4371-fe60-4375-9bcc-58d9bc90de22 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:05.779324 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa4371-fe60-4375-9bcc-58d9bc90de22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:05.779595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15aa4371-fe60-4375-9bcc-58d9bc90de22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:05.781221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-15aa4371-fe60-4375-9bcc-58d9bc90de22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:05.888476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa4371-fe60-4375-9bcc-58d9bc90de22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:05.889947 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15aa4371-fe60-4375-9bcc-58d9bc90de22 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:05.900633 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:05.908918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 659/1334] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:08.086338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870de0a-a433-4ff4-839a-f64ff089575a None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:08.086933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870de0a-a433-4ff4-839a-f64ff089575a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:08.086933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9870de0a-a433-4ff4-839a-f64ff089575a None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:08.088206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 676/1335] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:08.093335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708dc14-e699-406d-97fa-d664bb609c95 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:08.093719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708dc14-e699-406d-97fa-d664bb609c95 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:08.093878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c708dc14-e699-406d-97fa-d664bb609c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:08.159286 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c708dc14-e699-406d-97fa-d664bb609c95 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:08.160469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c708dc14-e699-406d-97fa-d664bb609c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:08.262951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 660/1336] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:08.556580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870de0a-a433-4ff4-839a-f64ff089575a None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:08.558139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9870de0a-a433-4ff4-839a-f64ff089575a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:08.598879 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:08.661658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9870de0a-a433-4ff4-839a-f64ff089575a None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:08.733091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec895c3-f098-4651-9778-0221699ac30f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:08.733683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec895c3-f098-4651-9778-0221699ac30f service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:08.733954 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec895c3-f098-4651-9778-0221699ac30f service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:08.734223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec895c3-f098-4651-9778-0221699ac30f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:08.735823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6ec895c3-f098-4651-9778-0221699ac30f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:08.815956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:08.842200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec895c3-f098-4651-9778-0221699ac30f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:08.843596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec895c3-f098-4651-9778-0221699ac30f service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:08.862573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 677/1337] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:11.344596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14539f2d-c553-4ee5-86a9-ea4c99a184fc None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:11.345155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14539f2d-c553-4ee5-86a9-ea4c99a184fc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:11.345270 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14539f2d-c553-4ee5-86a9-ea4c99a184fc None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:11.346545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 661/1338] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:11.352326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6638575-4752-4943-995f-6797e0ad04b4 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:11.352736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6638575-4752-4943-995f-6797e0ad04b4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:11.352906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6638575-4752-4943-995f-6797e0ad04b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:11.419838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-f6638575-4752-4943-995f-6797e0ad04b4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:11.421093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f6638575-4752-4943-995f-6797e0ad04b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:11.515728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 678/1339] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:11.812845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14539f2d-c553-4ee5-86a9-ea4c99a184fc None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:11.814366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-14539f2d-c553-4ee5-86a9-ea4c99a184fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:11.889606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:11.917634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14539f2d-c553-4ee5-86a9-ea4c99a184fc None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:12.006992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe13a0-bc41-43a4-bf43-cecb22ba8493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:12.007520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe13a0-bc41-43a4-bf43-cecb22ba8493 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:12.007794 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe13a0-bc41-43a4-bf43-cecb22ba8493 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:12.008061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe13a0-bc41-43a4-bf43-cecb22ba8493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:12.009771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bcfe13a0-bc41-43a4-bf43-cecb22ba8493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:12.118395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe13a0-bc41-43a4-bf43-cecb22ba8493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:12.119826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe13a0-bc41-43a4-bf43-cecb22ba8493 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:12.138972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 662/1340] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:16.245240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ccc9-1069-4bec-8f79-ccac4e0dc8f5 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:16.245742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ccc9-1069-4bec-8f79-ccac4e0dc8f5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:16.245742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2ccc9-1069-4bec-8f79-ccac4e0dc8f5 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:16.247034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 679/1341] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:16.252270 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdb275-d8c9-4b88-ba8f-fd72d8f1e3f9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:16.252803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdb275-d8c9-4b88-ba8f-fd72d8f1e3f9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:16.252903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bdb275-d8c9-4b88-ba8f-fd72d8f1e3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:16.258508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:16.326485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c6bdb275-d8c9-4b88-ba8f-fd72d8f1e3f9 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:16.327583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c6bdb275-d8c9-4b88-ba8f-fd72d8f1e3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:16.431060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 663/1342] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:16.723841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2ccc9-1069-4bec-8f79-ccac4e0dc8f5 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:16.726092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2ff2ccc9-1069-4bec-8f79-ccac4e0dc8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:16.776882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:16.835509 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2ccc9-1069-4bec-8f79-ccac4e0dc8f5 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:16.910049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80ba9bc-6ebb-4aca-8229-d6674cb86e52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:16.910655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ba9bc-6ebb-4aca-8229-d6674cb86e52 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:16.910932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ba9bc-6ebb-4aca-8229-d6674cb86e52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:16.911203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80ba9bc-6ebb-4aca-8229-d6674cb86e52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:16.913345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b80ba9bc-6ebb-4aca-8229-d6674cb86e52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:17.009408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:17.026936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ba9bc-6ebb-4aca-8229-d6674cb86e52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:17.028361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80ba9bc-6ebb-4aca-8229-d6674cb86e52 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:17.046490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 680/1343] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:19.835798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2b333-4c93-4341-b529-82a02b77c4e4 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:19.836337 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2b333-4c93-4341-b529-82a02b77c4e4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:19.836337 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b2b333-4c93-4341-b529-82a02b77c4e4 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:19.837596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 664/1344] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:19.842675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf80ec6-3718-47b0-8e1f-7284f01f490e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:19.843102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf80ec6-3718-47b0-8e1f-7284f01f490e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:19.843239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf80ec6-3718-47b0-8e1f-7284f01f490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:19.912308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-ecf80ec6-3718-47b0-8e1f-7284f01f490e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:19.913667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ecf80ec6-3718-47b0-8e1f-7284f01f490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:20.019512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 681/1345] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:19 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:20.301635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2b333-4c93-4341-b529-82a02b77c4e4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:20.303767 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-18b2b333-4c93-4341-b529-82a02b77c4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:20.406633 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:20.425062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b2b333-4c93-4341-b529-82a02b77c4e4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:20.499044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23af841e-7f0e-4fac-a420-5b0c704c1847 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:20.499638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af841e-7f0e-4fac-a420-5b0c704c1847 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:20.499916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af841e-7f0e-4fac-a420-5b0c704c1847 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:20.500183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23af841e-7f0e-4fac-a420-5b0c704c1847 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:20.501796 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-23af841e-7f0e-4fac-a420-5b0c704c1847 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:20.616499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af841e-7f0e-4fac-a420-5b0c704c1847 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:20.617979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23af841e-7f0e-4fac-a420-5b0c704c1847 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:20.636750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 665/1346] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:20 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:24.768165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d5ebf-9d6c-487e-a888-4b6586f5f566 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:24.768672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d5ebf-9d6c-487e-a888-4b6586f5f566 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:24.768898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3d5ebf-9d6c-487e-a888-4b6586f5f566 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:24.770187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 682/1347] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:24.775936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc695-a84d-4363-8278-ba802a21e215 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:24.776359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc695-a84d-4363-8278-ba802a21e215 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:24.776477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5dc695-a84d-4363-8278-ba802a21e215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:24.793060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:24.848409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3f5dc695-a84d-4363-8278-ba802a21e215 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:24.850288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3f5dc695-a84d-4363-8278-ba802a21e215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:24.952114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 666/1348] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:25.235277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d5ebf-9d6c-487e-a888-4b6586f5f566 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:25.236876 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-0a3d5ebf-9d6c-487e-a888-4b6586f5f566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:25.294122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:25.339421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3d5ebf-9d6c-487e-a888-4b6586f5f566 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:25.411072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdade34-95f2-4f2f-8589-4535088e8b9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:25.411574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdade34-95f2-4f2f-8589-4535088e8b9c service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:25.412486 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdade34-95f2-4f2f-8589-4535088e8b9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:25.412783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdade34-95f2-4f2f-8589-4535088e8b9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:25.414382 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3fdade34-95f2-4f2f-8589-4535088e8b9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:25.517224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:25.521929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdade34-95f2-4f2f-8589-4535088e8b9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:25.523429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdade34-95f2-4f2f-8589-4535088e8b9c service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:25.541861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 683/1349] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:30.551295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5240a6b-6706-4570-bd24-4cf1320671f6 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:30.551295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5240a6b-6706-4570-bd24-4cf1320671f6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:30.551295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5240a6b-6706-4570-bd24-4cf1320671f6 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:30.553420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 667/1350] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:35:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:30.562878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393ff8f-6005-40c8-a894-c7852ff9cd14 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:30.562878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393ff8f-6005-40c8-a894-c7852ff9cd14 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:30.562878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7393ff8f-6005-40c8-a894-c7852ff9cd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:30.663150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7393ff8f-6005-40c8-a894-c7852ff9cd14 None None] MFA Rules not processed for user `1d7e1b7dfa32448a88e0c90f10cdb2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:30.664725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7393ff8f-6005-40c8-a894-c7852ff9cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:30.809798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 684/1351] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:35:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:34.950302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=59 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:35.035358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b141b-5efb-4ce5-bb9a-0d43ddd962e8 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:35.035886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b141b-5efb-4ce5-bb9a-0d43ddd962e8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:35.036163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5b141b-5efb-4ce5-bb9a-0d43ddd962e8 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:35.038494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 668/1352] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:35:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:35.043385 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa623a7-740b-4145-8021-eb1f09882ac6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:35.044196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa623a7-740b-4145-8021-eb1f09882ac6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:35.044471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa623a7-740b-4145-8021-eb1f09882ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:35.101201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:35.126321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-9aa623a7-740b-4145-8021-eb1f09882ac6 None None] MFA Rules not processed for user `1d7e1b7dfa32448a88e0c90f10cdb2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:35.127635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9aa623a7-740b-4145-8021-eb1f09882ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:35.242956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 685/1353] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:35:35 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:36.735674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f080429-1477-436d-9602-50bba9eb2753 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:36.736200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f080429-1477-436d-9602-50bba9eb2753 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:36.736200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f080429-1477-436d-9602-50bba9eb2753 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:36.737826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 669/1354] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:35:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:36.743732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa1892-1602-4f18-b5a6-407c2cf16b75 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:36.744178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa1892-1602-4f18-b5a6-407c2cf16b75 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:36.744360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afa1892-1602-4f18-b5a6-407c2cf16b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:36.823498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-9afa1892-1602-4f18-b5a6-407c2cf16b75 None None] MFA Rules not processed for user `1d7e1b7dfa32448a88e0c90f10cdb2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:36.824695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9afa1892-1602-4f18-b5a6-407c2cf16b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:36.867892 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:36.942859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 686/1355] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:35:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:35:40.390782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fce0a9-834d-439a-a6de-36a91e2cf722 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:40.392008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fce0a9-834d-439a-a6de-36a91e2cf722 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:40.392978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fce0a9-834d-439a-a6de-36a91e2cf722 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:40.395877 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 670/1356] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:40.400961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b019d32-bfee-495d-8100-bd186ea60991 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:40.401791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b019d32-bfee-495d-8100-bd186ea60991 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:40.402208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b019d32-bfee-495d-8100-bd186ea60991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:40.487931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-4b019d32-bfee-495d-8100-bd186ea60991 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:40.489169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4b019d32-bfee-495d-8100-bd186ea60991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:40.600545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 687/1357] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:40.909588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143f69-bb28-4210-a955-5fcac199a2ea None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:40.910270 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143f69-bb28-4210-a955-5fcac199a2ea None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:40.910879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e143f69-bb28-4210-a955-5fcac199a2ea None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:40.914168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 671/1358] 10.222.0.39 () {54 vars in 962 bytes} [Tue Sep 6 22:35:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:40.921899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415a4e8-4f96-4093-966f-5507300fac30 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:40.922897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415a4e8-4f96-4093-966f-5507300fac30 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:40.923520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1415a4e8-4f96-4093-966f-5507300fac30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:40.942723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:41.011207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-1415a4e8-4f96-4093-966f-5507300fac30 None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:41.012726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1415a4e8-4f96-4093-966f-5507300fac30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:41.141628 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 688/1359] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Sep 6 22:35:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:35:41.149477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e143f69-bb28-4210-a955-5fcac199a2ea None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:41.150965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2e143f69-bb28-4210-a955-5fcac199a2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:41.307311 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e143f69-bb28-4210-a955-5fcac199a2ea None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:41.385910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:41.428117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b167d1-4543-4f0a-b1d3-50067618e677 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:41.431400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b167d1-4543-4f0a-b1d3-50067618e677 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:41.431400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b167d1-4543-4f0a-b1d3-50067618e677 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:41.435477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b167d1-4543-4f0a-b1d3-50067618e677 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:41.437258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a9b167d1-4543-4f0a-b1d3-50067618e677 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:41.536546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b167d1-4543-4f0a-b1d3-50067618e677 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:41.538729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b167d1-4543-4f0a-b1d3-50067618e677 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:41.558702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 672/1360] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:43.982705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc380f-2bf0-4062-8ada-486986bccdf6 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:43.983215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc380f-2bf0-4062-8ada-486986bccdf6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:43.983372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbc380f-2bf0-4062-8ada-486986bccdf6 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:43.984733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 689/1361] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:43.992816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d201d49-9100-423b-b86f-17d5bcc7defb None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:43.992816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d201d49-9100-423b-b86f-17d5bcc7defb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:43.992816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d201d49-9100-423b-b86f-17d5bcc7defb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:44.065812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4d201d49-9100-423b-b86f-17d5bcc7defb None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:44.067118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4d201d49-9100-423b-b86f-17d5bcc7defb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:44.105137 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:44.168413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 673/1362] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:44.458960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc380f-2bf0-4062-8ada-486986bccdf6 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:44.460559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6bbc380f-2bf0-4062-8ada-486986bccdf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:44.537986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:44.653016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbc380f-2bf0-4062-8ada-486986bccdf6 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:44.784120 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e942c9-b18d-49f8-942b-0790efecaa28 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:44.784666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e942c9-b18d-49f8-942b-0790efecaa28 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:44.784966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e942c9-b18d-49f8-942b-0790efecaa28 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:44.785235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e942c9-b18d-49f8-942b-0790efecaa28 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:44.786824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-31e942c9-b18d-49f8-942b-0790efecaa28 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:44.809760 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:44.880567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e942c9-b18d-49f8-942b-0790efecaa28 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:44.882498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e942c9-b18d-49f8-942b-0790efecaa28 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:44.902020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 690/1363] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:47.586515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a873b42-9884-4a5e-bf26-a1007a4e4c05 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:47.587076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a873b42-9884-4a5e-bf26-a1007a4e4c05 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:47.587076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a873b42-9884-4a5e-bf26-a1007a4e4c05 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:47.588571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 674/1364] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:47.593733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11822c-fa36-4e06-ac1f-d06ea3bd165d None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:47.594139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11822c-fa36-4e06-ac1f-d06ea3bd165d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:47.594306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a11822c-fa36-4e06-ac1f-d06ea3bd165d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:47.662884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2a11822c-fa36-4e06-ac1f-d06ea3bd165d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:47.664978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2a11822c-fa36-4e06-ac1f-d06ea3bd165d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:47.764970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:47.768858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 691/1365] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:48.059691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2910b4-b60e-49a3-bf1d-5cd0f451713c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:48.060101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2910b4-b60e-49a3-bf1d-5cd0f451713c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:48.060477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2910b4-b60e-49a3-bf1d-5cd0f451713c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:48.062061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 675/1366] 10.222.0.39 () {54 vars in 962 bytes} [Tue Sep 6 22:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:48.068599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ee564-d9b5-4cbd-83fa-da081f5d191a None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:48.068940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ee564-d9b5-4cbd-83fa-da081f5d191a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:48.069190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96ee564-d9b5-4cbd-83fa-da081f5d191a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:48.149329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c96ee564-d9b5-4cbd-83fa-da081f5d191a None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:48.150532 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c96ee564-d9b5-4cbd-83fa-da081f5d191a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:48.274907 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 692/1367] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Sep 6 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:35:48.282685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2910b4-b60e-49a3-bf1d-5cd0f451713c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:48.284384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2a2910b4-b60e-49a3-bf1d-5cd0f451713c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:48.448529 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2910b4-b60e-49a3-bf1d-5cd0f451713c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:48.476402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:48.581762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c228daf2-24aa-4839-b51f-c0cdb8d6d119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:48.582483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228daf2-24aa-4839-b51f-c0cdb8d6d119 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:48.582865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228daf2-24aa-4839-b51f-c0cdb8d6d119 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:48.583243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c228daf2-24aa-4839-b51f-c0cdb8d6d119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:48.585032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c228daf2-24aa-4839-b51f-c0cdb8d6d119 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:48.685588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228daf2-24aa-4839-b51f-c0cdb8d6d119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:48.687186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c228daf2-24aa-4839-b51f-c0cdb8d6d119 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:48.706281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 676/1368] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:48.751026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a8d6f-f5ea-4911-971d-113a9d0c059c None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:48.751602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a8d6f-f5ea-4911-971d-113a9d0c059c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:48.751602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a8d6f-f5ea-4911-971d-113a9d0c059c None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:48.753098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 693/1369] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:48.758687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb736c-bb76-4c31-8572-58595bb2e36c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:48.758972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb736c-bb76-4c31-8572-58595bb2e36c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:48.759249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cb736c-bb76-4c31-8572-58595bb2e36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:48.798758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:48.841543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-90cb736c-bb76-4c31-8572-58595bb2e36c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:48.842767 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-90cb736c-bb76-4c31-8572-58595bb2e36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:48.980521 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 677/1370] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:49.316724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144a026-6319-4d95-b1d6-8c81ae50acd0 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:49.317316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144a026-6319-4d95-b1d6-8c81ae50acd0 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:49.318126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4144a026-6319-4d95-b1d6-8c81ae50acd0 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:49.320939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 694/1371] 10.222.0.39 () {54 vars in 964 bytes} [Tue Sep 6 22:35:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:49.326374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95f7c6-33d8-4f0a-b0c2-d85c5b1611cc None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:49.326734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95f7c6-33d8-4f0a-b0c2-d85c5b1611cc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:49.327104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c95f7c6-33d8-4f0a-b0c2-d85c5b1611cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:49.414569 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-9c95f7c6-33d8-4f0a-b0c2-d85c5b1611cc None None] MFA Rules not processed for user `d2da9e23277a4de2a61b9727737ab8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:49.415794 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9c95f7c6-33d8-4f0a-b0c2-d85c5b1611cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:49.465595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:49.500825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 678/1372] 10.222.0.39 () {58 vars in 1061 bytes} [Tue Sep 6 22:35:49 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:49.509398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144a026-6319-4d95-b1d6-8c81ae50acd0 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:49.511008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4144a026-6319-4d95-b1d6-8c81ae50acd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:49.576445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:49.650405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4144a026-6319-4d95-b1d6-8c81ae50acd0 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:49.743194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e15fea3-7786-4fdc-b04f-a66ae3c93d66 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:49.743939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15fea3-7786-4fdc-b04f-a66ae3c93d66 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:49.744319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15fea3-7786-4fdc-b04f-a66ae3c93d66 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:49.744713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e15fea3-7786-4fdc-b04f-a66ae3c93d66 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:49.746786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9e15fea3-7786-4fdc-b04f-a66ae3c93d66 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:49.851359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:49.865301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15fea3-7786-4fdc-b04f-a66ae3c93d66 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:49.866868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e15fea3-7786-4fdc-b04f-a66ae3c93d66 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:49.886942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 695/1373] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:50.967221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317df26-2306-4c11-82f5-1411c6a02f2f None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:50.967742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317df26-2306-4c11-82f5-1411c6a02f2f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:50.967742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b317df26-2306-4c11-82f5-1411c6a02f2f None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:50.969352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 679/1374] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:50.975218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebba3ef-faa2-454b-9475-7bf203576744 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:50.975709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebba3ef-faa2-454b-9475-7bf203576744 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:50.975868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebba3ef-faa2-454b-9475-7bf203576744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:51.042883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-5ebba3ef-faa2-454b-9475-7bf203576744 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:51.044095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5ebba3ef-faa2-454b-9475-7bf203576744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:51.141636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 696/1375] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:51.435936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317df26-2306-4c11-82f5-1411c6a02f2f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:51.436835 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b317df26-2306-4c11-82f5-1411c6a02f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:51.594990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b317df26-2306-4c11-82f5-1411c6a02f2f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:51.649212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:51.712967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25910409-58a6-4da8-8da6-43113933c841 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:51.713580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25910409-58a6-4da8-8da6-43113933c841 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:51.713858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25910409-58a6-4da8-8da6-43113933c841 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:51.714129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25910409-58a6-4da8-8da6-43113933c841 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:51.715761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-25910409-58a6-4da8-8da6-43113933c841 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:51.813004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25910409-58a6-4da8-8da6-43113933c841 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:51.815255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25910409-58a6-4da8-8da6-43113933c841 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:51.835200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 680/1376] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:54.110932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2fb11-6b44-4ce5-b764-70d0bc801ffb None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:54.111501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2fb11-6b44-4ce5-b764-70d0bc801ffb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:54.111501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead2fb11-6b44-4ce5-b764-70d0bc801ffb None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:54.115023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 697/1377] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:54.121416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f46b1-82b4-4904-ba21-f13e333f10cd None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:54.121859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f46b1-82b4-4904-ba21-f13e333f10cd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:54.121954 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f46b1-82b4-4904-ba21-f13e333f10cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:54.200197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-958f46b1-82b4-4904-ba21-f13e333f10cd None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:54.201596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-958f46b1-82b4-4904-ba21-f13e333f10cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:54.208955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:54.312847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 681/1378] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:54.599680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2fb11-6b44-4ce5-b764-70d0bc801ffb None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:54.601359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ead2fb11-6b44-4ce5-b764-70d0bc801ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:54.663969 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:54.759746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead2fb11-6b44-4ce5-b764-70d0bc801ffb None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:54.889742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908d75c-493e-4ffe-afde-970e2b8330da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:54.890343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908d75c-493e-4ffe-afde-970e2b8330da service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:54.890614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908d75c-493e-4ffe-afde-970e2b8330da service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:54.890882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908d75c-493e-4ffe-afde-970e2b8330da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:54.892861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a908d75c-493e-4ffe-afde-970e2b8330da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:54.915562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:55.003340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908d75c-493e-4ffe-afde-970e2b8330da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:55.005112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a908d75c-493e-4ffe-afde-970e2b8330da service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:55.036940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 698/1379] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:35:57.235218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854eece-014b-4bc0-82de-58ff9624824a None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:57.235993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854eece-014b-4bc0-82de-58ff9624824a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:57.236170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a854eece-014b-4bc0-82de-58ff9624824a None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:57.237966 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 682/1380] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:35:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:35:57.248956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72642d3-1527-45d4-876b-f7629d932338 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:57.248956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72642d3-1527-45d4-876b-f7629d932338 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:57.248956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72642d3-1527-45d4-876b-f7629d932338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:57.313752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-b72642d3-1527-45d4-876b-f7629d932338 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:35:57.315001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b72642d3-1527-45d4-876b-f7629d932338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:57.420732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:57.427602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 699/1381] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:35:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:35:57.734647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854eece-014b-4bc0-82de-58ff9624824a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:35:57.736160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a854eece-014b-4bc0-82de-58ff9624824a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:57.895762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:57.908460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a854eece-014b-4bc0-82de-58ff9624824a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:35:58.059991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751710b3-c9a2-43a3-9a1e-664088f16e2d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:35:58.061284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751710b3-c9a2-43a3-9a1e-664088f16e2d service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:35:58.061715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751710b3-c9a2-43a3-9a1e-664088f16e2d service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:35:58.062210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751710b3-c9a2-43a3-9a1e-664088f16e2d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:35:58.065142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-751710b3-c9a2-43a3-9a1e-664088f16e2d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:35:58.166925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751710b3-c9a2-43a3-9a1e-664088f16e2d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:35:58.168622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751710b3-c9a2-43a3-9a1e-664088f16e2d service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:35:58.178935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:35:58.188501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 683/1382] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:35:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:01.256310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55666501-3663-4bff-8696-f4e27f513f24 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:01.257377 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55666501-3663-4bff-8696-f4e27f513f24 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:01.257492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55666501-3663-4bff-8696-f4e27f513f24 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:01.258980 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 700/1383] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:36:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:01.264746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84161888-63f8-453f-9236-26f75db2942d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:01.265219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84161888-63f8-453f-9236-26f75db2942d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:01.265316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84161888-63f8-453f-9236-26f75db2942d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:01.347437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-84161888-63f8-453f-9236-26f75db2942d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:01.349045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-84161888-63f8-453f-9236-26f75db2942d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:01.461924 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 684/1384] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:36:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:01.769335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55666501-3663-4bff-8696-f4e27f513f24 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:01.770878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-55666501-3663-4bff-8696-f4e27f513f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:01.932125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55666501-3663-4bff-8696-f4e27f513f24 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:02.006771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:02.059175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83999dec-2ff6-4b19-8eb8-d12f4572f60b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:02.059771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83999dec-2ff6-4b19-8eb8-d12f4572f60b service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:02.060054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83999dec-2ff6-4b19-8eb8-d12f4572f60b service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:02.060335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83999dec-2ff6-4b19-8eb8-d12f4572f60b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:02.062042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-83999dec-2ff6-4b19-8eb8-d12f4572f60b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:02.156736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83999dec-2ff6-4b19-8eb8-d12f4572f60b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:02.158710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83999dec-2ff6-4b19-8eb8-d12f4572f60b service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:02.177001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 701/1385] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:04.388783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8779db0-354c-49bd-acf2-d29abc06f757 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:04.389395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8779db0-354c-49bd-acf2-d29abc06f757 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:04.389395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8779db0-354c-49bd-acf2-d29abc06f757 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:04.390722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 685/1386] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:36:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:04.396239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b21ac-ec7f-4764-8a96-9d82a0077270 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:04.396648 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b21ac-ec7f-4764-8a96-9d82a0077270 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:04.396826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b21ac-ec7f-4764-8a96-9d82a0077270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:04.463219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-4c5b21ac-ec7f-4764-8a96-9d82a0077270 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:04.464342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4c5b21ac-ec7f-4764-8a96-9d82a0077270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:04.497684 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:04.566378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 702/1387] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:36:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:04.861535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8779db0-354c-49bd-acf2-d29abc06f757 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:04.863049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b8779db0-354c-49bd-acf2-d29abc06f757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:04.946200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:05.016624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8779db0-354c-49bd-acf2-d29abc06f757 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:05.135990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c63e7d-ca20-4932-92be-ff1be2ec5d58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:05.136513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63e7d-ca20-4932-92be-ff1be2ec5d58 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:05.136817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63e7d-ca20-4932-92be-ff1be2ec5d58 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:05.137094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c63e7d-ca20-4932-92be-ff1be2ec5d58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:05.138704 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-98c63e7d-ca20-4932-92be-ff1be2ec5d58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:05.176904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:05.180361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace32c4-89ce-46a7-9ef3-352972184a78 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:05.180649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace32c4-89ce-46a7-9ef3-352972184a78 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:05.180949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ace32c4-89ce-46a7-9ef3-352972184a78 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:05.182420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 703/1388] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:36:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:36:05.187425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8102ff-ee04-4ee9-8982-312c41cee2ef None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:05.187704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8102ff-ee04-4ee9-8982-312c41cee2ef None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:05.187977 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8102ff-ee04-4ee9-8982-312c41cee2ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:05.231960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c63e7d-ca20-4932-92be-ff1be2ec5d58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:05.233587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c63e7d-ca20-4932-92be-ff1be2ec5d58 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:05.251797 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 686/1389] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:05.254821 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-cd8102ff-ee04-4ee9-8982-312c41cee2ef None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:05.256008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-cd8102ff-ee04-4ee9-8982-312c41cee2ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:05.363589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 704/1390] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:36:05.675400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4c0ec-5d06-400a-ac20-7c545ee59e88 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:05.675870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4c0ec-5d06-400a-ac20-7c545ee59e88 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:05.676136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a4c0ec-5d06-400a-ac20-7c545ee59e88 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:05.677569 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 687/1391] 10.222.0.39 () {54 vars in 961 bytes} [Tue Sep 6 22:36:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:05.683256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dac70a-960a-4bda-b9af-aad15b1fc0f6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:05.683696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dac70a-960a-4bda-b9af-aad15b1fc0f6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:05.683805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dac70a-960a-4bda-b9af-aad15b1fc0f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:05.699978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:05.761730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-55dac70a-960a-4bda-b9af-aad15b1fc0f6 None None] MFA Rules not processed for user `42cf265450454ff9a27a29093336205e`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:05.762968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-55dac70a-960a-4bda-b9af-aad15b1fc0f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:05.839426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 705/1392] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Sep 6 22:36:05 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:36:05.848371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4c0ec-5d06-400a-ac20-7c545ee59e88 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:05.849905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-53a4c0ec-5d06-400a-ac20-7c545ee59e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:05.955107 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a4c0ec-5d06-400a-ac20-7c545ee59e88 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:06.014339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:06.030811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ad2060-1c10-4d1f-b48d-bcc3d20ce666 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42cf265450454ff9a27a29093336205e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:06.031409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad2060-1c10-4d1f-b48d-bcc3d20ce666 service glance] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:06.031681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad2060-1c10-4d1f-b48d-bcc3d20ce666 service glance] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:06.031949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ad2060-1c10-4d1f-b48d-bcc3d20ce666 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:06.033570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-32ad2060-1c10-4d1f-b48d-bcc3d20ce666 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:06.143735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad2060-1c10-4d1f-b48d-bcc3d20ce666 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:06.145251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ad2060-1c10-4d1f-b48d-bcc3d20ce666 service glance] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:06.165657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 688/1393] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:06.910609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdd357-23d3-4564-9080-0c99746e0f49 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:06.911001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdd357-23d3-4564-9080-0c99746e0f49 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:06.911161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becdd357-23d3-4564-9080-0c99746e0f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:06.980621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-becdd357-23d3-4564-9080-0c99746e0f49 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:06.981906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-becdd357-23d3-4564-9080-0c99746e0f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:07.039289 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:07.071687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 706/1394] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:36:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:07.114939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078d1b8-3aa7-4402-b8f1-8dba6b5ea435 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:07.115412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078d1b8-3aa7-4402-b8f1-8dba6b5ea435 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:07.115534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078d1b8-3aa7-4402-b8f1-8dba6b5ea435 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:07.116890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 689/1395] 10.222.0.39 () {54 vars in 943 bytes} [Tue Sep 6 22:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:07.123446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24625aa6-1746-4245-96e7-cd3cfc9d97dd None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:07.123735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24625aa6-1746-4245-96e7-cd3cfc9d97dd None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:07.124013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24625aa6-1746-4245-96e7-cd3cfc9d97dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:07.202591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-24625aa6-1746-4245-96e7-cd3cfc9d97dd None None] MFA Rules not processed for user `a3f142a9e141440cbc854ade4f4135a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:07.203719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-24625aa6-1746-4245-96e7-cd3cfc9d97dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:07.310324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 707/1396] 10.222.0.39 () {58 vars in 1040 bytes} [Tue Sep 6 22:36:07 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:07.320318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078d1b8-3aa7-4402-b8f1-8dba6b5ea435 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:07.321869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f078d1b8-3aa7-4402-b8f1-8dba6b5ea435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:07.417721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:07.468148 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078d1b8-3aa7-4402-b8f1-8dba6b5ea435 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:07.468711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a65a-cf0c-4d9c-bcf4-0f43d8582624 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:07.469060 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a65a-cf0c-4d9c-bcf4-0f43d8582624 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:07.469333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9891a65a-cf0c-4d9c-bcf4-0f43d8582624 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:07.471028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 708/1397] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:36:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:36:07.475714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbad90-7169-44e3-9471-9890a2710337 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:07.475999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbad90-7169-44e3-9471-9890a2710337 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:07.476266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fbad90-7169-44e3-9471-9890a2710337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:07.512640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:07.543924 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-45fbad90-7169-44e3-9471-9890a2710337 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:07.545164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-45fbad90-7169-44e3-9471-9890a2710337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:07.576965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7aad1c1-d6b6-4b6f-883c-3c72ad53e52b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:07.577119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad1c1-d6b6-4b6f-883c-3c72ad53e52b service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:07.577287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad1c1-d6b6-4b6f-883c-3c72ad53e52b service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:07.577559 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7aad1c1-d6b6-4b6f-883c-3c72ad53e52b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:07.579214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d7aad1c1-d6b6-4b6f-883c-3c72ad53e52b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:07.646993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 709/1398] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:36:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:36:07.653540 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:07.666565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aad1c1-d6b6-4b6f-883c-3c72ad53e52b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:07.668193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7aad1c1-d6b6-4b6f-883c-3c72ad53e52b service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:07.676176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 690/1399] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:36:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:36:07.946472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbad90-7169-44e3-9471-9890a2710337 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:07.948209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-45fbad90-7169-44e3-9471-9890a2710337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:08.136737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:08.142646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fbad90-7169-44e3-9471-9890a2710337 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:08.301789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e615133-93a6-4da4-8120-fd06d3c3b28c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:08.302403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e615133-93a6-4da4-8120-fd06d3c3b28c service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:08.302673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e615133-93a6-4da4-8120-fd06d3c3b28c service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:08.302939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e615133-93a6-4da4-8120-fd06d3c3b28c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:08.304729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5e615133-93a6-4da4-8120-fd06d3c3b28c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:08.406950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e615133-93a6-4da4-8120-fd06d3c3b28c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:08.408530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e615133-93a6-4da4-8120-fd06d3c3b28c service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:08.430055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 710/1400] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:36:10.656206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dfde4-bd0e-4bf1-85fd-0cc049a339f7 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:10.656206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dfde4-bd0e-4bf1-85fd-0cc049a339f7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:10.656206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9dfde4-bd0e-4bf1-85fd-0cc049a339f7 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:10.657206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 691/1401] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:36:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:10.662819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9c26-9b22-4bc4-b83e-2c23f0aa9ff0 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:10.663302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9c26-9b22-4bc4-b83e-2c23f0aa9ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:10.663567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a9c26-9b22-4bc4-b83e-2c23f0aa9ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:10.667957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:10.737803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2e6a9c26-9b22-4bc4-b83e-2c23f0aa9ff0 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:10.739387 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2e6a9c26-9b22-4bc4-b83e-2c23f0aa9ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:10.890154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 711/1402] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:36:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:11.183004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dfde4-bd0e-4bf1-85fd-0cc049a339f7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:11.184523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6a9dfde4-bd0e-4bf1-85fd-0cc049a339f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:11.348908 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9dfde4-bd0e-4bf1-85fd-0cc049a339f7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:11.431545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:11.517911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb70dc-f9a3-47c0-b2f8-2abc3a19c417 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:11.519002 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb70dc-f9a3-47c0-b2f8-2abc3a19c417 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:11.519499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb70dc-f9a3-47c0-b2f8-2abc3a19c417 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:11.519996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb70dc-f9a3-47c0-b2f8-2abc3a19c417 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:11.522167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-10bb70dc-f9a3-47c0-b2f8-2abc3a19c417 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:11.637483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb70dc-f9a3-47c0-b2f8-2abc3a19c417 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:11.642694 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb70dc-f9a3-47c0-b2f8-2abc3a19c417 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:11.660105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 692/1403] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:13.985547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06700a4b-18d6-46c4-a28e-67eb25adc200 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:13.986149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06700a4b-18d6-46c4-a28e-67eb25adc200 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:13.986149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06700a4b-18d6-46c4-a28e-67eb25adc200 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:13.987414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 712/1404] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:36:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:13.992906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57540a-a684-44c3-8e5d-a092562c9680 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:13.993360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57540a-a684-44c3-8e5d-a092562c9680 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:13.993485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c57540a-a684-44c3-8e5d-a092562c9680 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:14.077255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7c57540a-a684-44c3-8e5d-a092562c9680 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:14.078479 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7c57540a-a684-44c3-8e5d-a092562c9680 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:14.110555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:14.193637 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 693/1405] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:36:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:14.486764 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06700a4b-18d6-46c4-a28e-67eb25adc200 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:14.488294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-06700a4b-18d6-46c4-a28e-67eb25adc200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:14.589837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:14.660098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06700a4b-18d6-46c4-a28e-67eb25adc200 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:14.792354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b76ea-6765-4e46-9a21-034ac2af7183 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:14.793002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b76ea-6765-4e46-9a21-034ac2af7183 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:14.793355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b76ea-6765-4e46-9a21-034ac2af7183 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:14.793627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b76ea-6765-4e46-9a21-034ac2af7183 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:14.795233 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a14b76ea-6765-4e46-9a21-034ac2af7183 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:14.841849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:14.893485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b76ea-6765-4e46-9a21-034ac2af7183 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:14.895096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b76ea-6765-4e46-9a21-034ac2af7183 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:14.913929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 713/1406] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:17.143688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a78d8-02b2-41d4-a51f-7ef69132a4ff None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:17.144224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a78d8-02b2-41d4-a51f-7ef69132a4ff None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:17.144224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9a78d8-02b2-41d4-a51f-7ef69132a4ff None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:17.145509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 694/1407] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:36:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:17.151299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c9759-9a19-4d29-9751-384d594b4a49 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:17.151730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c9759-9a19-4d29-9751-384d594b4a49 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:17.151837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541c9759-9a19-4d29-9751-384d594b4a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:17.233022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-541c9759-9a19-4d29-9751-384d594b4a49 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:17.234647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-541c9759-9a19-4d29-9751-384d594b4a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:17.389299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 714/1408] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:17.670035 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a78d8-02b2-41d4-a51f-7ef69132a4ff None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:17.671620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8c9a78d8-02b2-41d4-a51f-7ef69132a4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:17.970656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9a78d8-02b2-41d4-a51f-7ef69132a4ff None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:17.982427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:18.096520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6be59f-f039-4946-8e51-80471a488eac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:18.097599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be59f-f039-4946-8e51-80471a488eac service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:18.098165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be59f-f039-4946-8e51-80471a488eac service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:18.098433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6be59f-f039-4946-8e51-80471a488eac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:18.100072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0a6be59f-f039-4946-8e51-80471a488eac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:18.207642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6be59f-f039-4946-8e51-80471a488eac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:18.209308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6be59f-f039-4946-8e51-80471a488eac service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:18.229413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 695/1409] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 560 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:27.374719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=91 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:28.226530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=94 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:30.531644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c360e-e301-479f-b395-60deebcb5d68 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:30.532208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c360e-e301-479f-b395-60deebcb5d68 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:30.532208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41c360e-e301-479f-b395-60deebcb5d68 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:30.533576 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 715/1410] 10.222.0.39 () {54 vars in 918 bytes} [Tue Sep 6 22:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:30.539374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221825-07f5-485a-aeb0-825e59c0134d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:30.539808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221825-07f5-485a-aeb0-825e59c0134d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:30.539921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4221825-07f5-485a-aeb0-825e59c0134d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:30.615392 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a4221825-07f5-485a-aeb0-825e59c0134d None None] MFA Rules not processed for user `1d7e1b7dfa32448a88e0c90f10cdb2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:30.616839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a4221825-07f5-485a-aeb0-825e59c0134d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:30.722220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 696/1411] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Sep 6 22:36:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:30.736932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c06999-8f5e-41c1-b342-d5eb9050957b None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:30.737215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c06999-8f5e-41c1-b342-d5eb9050957b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:30.737487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c06999-8f5e-41c1-b342-d5eb9050957b None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:30.738815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 716/1412] 10.222.0.39 () {54 vars in 918 bytes} [Tue Sep 6 22:36:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:30.743935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cffea-a3a4-464a-9152-69dbee997ed0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:30.744217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cffea-a3a4-464a-9152-69dbee997ed0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:30.744489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62cffea-a3a4-464a-9152-69dbee997ed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:30.819343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:30.822119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e62cffea-a3a4-464a-9152-69dbee997ed0 None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:30.823839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e62cffea-a3a4-464a-9152-69dbee997ed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:30.937223 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 697/1413] 10.222.0.39 () {58 vars in 1015 bytes} [Tue Sep 6 22:36:30 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:33.495125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de54e12-b761-43e2-8c44-32efc191f38e None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:33.495665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de54e12-b761-43e2-8c44-32efc191f38e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:33.495665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de54e12-b761-43e2-8c44-32efc191f38e None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:33.497056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 717/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:36:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:33.502164 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46a981-437b-48c2-9834-f23bb706ff24 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:33.502558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46a981-437b-48c2-9834-f23bb706ff24 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:33.502697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf46a981-437b-48c2-9834-f23bb706ff24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:33.569039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-bf46a981-437b-48c2-9834-f23bb706ff24 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:33.570277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bf46a981-437b-48c2-9834-f23bb706ff24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:33.624124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:33.666030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 698/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:36:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:33.954476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de54e12-b761-43e2-8c44-32efc191f38e None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:33.956156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5de54e12-b761-43e2-8c44-32efc191f38e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:34.098562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de54e12-b761-43e2-8c44-32efc191f38e None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:34.170006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:34.210367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573a9f89-a331-43ef-b8f4-7c22ef5d7840 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:34.210894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9f89-a331-43ef-b8f4-7c22ef5d7840 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:34.211169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9f89-a331-43ef-b8f4-7c22ef5d7840 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:34.211436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573a9f89-a331-43ef-b8f4-7c22ef5d7840 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:34.213068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-573a9f89-a331-43ef-b8f4-7c22ef5d7840 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:34.301132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a9f89-a331-43ef-b8f4-7c22ef5d7840 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:34.303287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573a9f89-a331-43ef-b8f4-7c22ef5d7840 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:34.321046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 718/1416] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:35.461386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827561c-f258-4030-a11c-87e6c7340cdb None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:35.461952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827561c-f258-4030-a11c-87e6c7340cdb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:35.461952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8827561c-f258-4030-a11c-87e6c7340cdb None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:35.463279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 699/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:36:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:35.469435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be86d9-f156-496c-b3ff-b9fb75212099 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:35.469855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be86d9-f156-496c-b3ff-b9fb75212099 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:35.469981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50be86d9-f156-496c-b3ff-b9fb75212099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:35.544802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-50be86d9-f156-496c-b3ff-b9fb75212099 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:35.545998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-50be86d9-f156-496c-b3ff-b9fb75212099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:35.573373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:35.643468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 719/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:36:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:35.793119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827561c-f258-4030-a11c-87e6c7340cdb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:35.794553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8827561c-f258-4030-a11c-87e6c7340cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:35.938616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8827561c-f258-4030-a11c-87e6c7340cdb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:35.977501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:36.049512 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9c0035-107c-4dff-aa11-8962a5e73917 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:36.050032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0035-107c-4dff-aa11-8962a5e73917 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:36.050301 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0035-107c-4dff-aa11-8962a5e73917 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:36.050559 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9c0035-107c-4dff-aa11-8962a5e73917 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:36.052134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-de9c0035-107c-4dff-aa11-8962a5e73917 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:36.137566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c0035-107c-4dff-aa11-8962a5e73917 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:36.139121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9c0035-107c-4dff-aa11-8962a5e73917 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:36.157473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 700/1419] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:37.288791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e7425-d5ec-493a-a502-d5e7bb5c1ceb None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:37.289367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e7425-d5ec-493a-a502-d5e7bb5c1ceb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:37.289367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5e7425-d5ec-493a-a502-d5e7bb5c1ceb None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:37.290606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 720/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:36:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:36:37.296783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44e648-6ca4-492b-a00e-03254b6c9a82 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:37.297227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44e648-6ca4-492b-a00e-03254b6c9a82 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:37.297325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa44e648-6ca4-492b-a00e-03254b6c9a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:37.350574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:37.372498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-fa44e648-6ca4-492b-a00e-03254b6c9a82 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:37.373720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fa44e648-6ca4-492b-a00e-03254b6c9a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:37.467904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 701/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:36:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:37.619985 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e7425-d5ec-493a-a502-d5e7bb5c1ceb None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:37.621550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7b5e7425-d5ec-493a-a502-d5e7bb5c1ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:37.766146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5e7425-d5ec-493a-a502-d5e7bb5c1ceb None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:37.779104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:37.877630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522120e3-7bb4-44b0-8349-135f3de0401a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:37.878229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522120e3-7bb4-44b0-8349-135f3de0401a service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:37.878498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522120e3-7bb4-44b0-8349-135f3de0401a service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:37.878761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522120e3-7bb4-44b0-8349-135f3de0401a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:37.880337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-522120e3-7bb4-44b0-8349-135f3de0401a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:37.969174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522120e3-7bb4-44b0-8349-135f3de0401a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:37.970746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522120e3-7bb4-44b0-8349-135f3de0401a service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:37.988937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 721/1422] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:39.142857 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9741aa-09c1-468d-9cd9-8dd0c7ef5baa None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:39.143390 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9741aa-09c1-468d-9cd9-8dd0c7ef5baa None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:39.143390 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9741aa-09c1-468d-9cd9-8dd0c7ef5baa None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:39.144742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 702/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:36:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:39.151046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775b699-a40a-4809-99ce-387e66ec064e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:39.151487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775b699-a40a-4809-99ce-387e66ec064e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:39.151583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775b699-a40a-4809-99ce-387e66ec064e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:39.170735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:39.228187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-3775b699-a40a-4809-99ce-387e66ec064e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:39.229361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3775b699-a40a-4809-99ce-387e66ec064e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:39.325830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 722/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:36:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:39.476577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9741aa-09c1-468d-9cd9-8dd0c7ef5baa None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:39.478139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-da9741aa-09c1-468d-9cd9-8dd0c7ef5baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:39.589316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:39.625169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9741aa-09c1-468d-9cd9-8dd0c7ef5baa None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:39.734397 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e40ecb-2fa4-457b-9a83-c72cab5a365b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:39.734989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e40ecb-2fa4-457b-9a83-c72cab5a365b service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:39.735258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e40ecb-2fa4-457b-9a83-c72cab5a365b service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:39.735519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e40ecb-2fa4-457b-9a83-c72cab5a365b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:39.737131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-50e40ecb-2fa4-457b-9a83-c72cab5a365b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:39.811009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:39.825514 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e40ecb-2fa4-457b-9a83-c72cab5a365b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:39.827146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e40ecb-2fa4-457b-9a83-c72cab5a365b service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:39.844740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 703/1425] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:40.949974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c811e6c-eab0-41fd-b25c-c7b43594f936 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:40.950480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c811e6c-eab0-41fd-b25c-c7b43594f936 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:40.950480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c811e6c-eab0-41fd-b25c-c7b43594f936 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:40.951846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 723/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:40.958067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ee1d3-9fc5-4023-8703-1b50187026b5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:40.958497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ee1d3-9fc5-4023-8703-1b50187026b5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:40.959275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4ee1d3-9fc5-4023-8703-1b50187026b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:41.034466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6a4ee1d3-9fc5-4023-8703-1b50187026b5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:41.035662 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6a4ee1d3-9fc5-4023-8703-1b50187026b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:41.129242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 704/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:36:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:41.278143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c811e6c-eab0-41fd-b25c-c7b43594f936 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:41.279595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3c811e6c-eab0-41fd-b25c-c7b43594f936 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:41.362408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:41.423350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c811e6c-eab0-41fd-b25c-c7b43594f936 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:41.533998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a934f2ac-69af-447a-affb-2e2277d5cd78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:41.534522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934f2ac-69af-447a-affb-2e2277d5cd78 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:41.534790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934f2ac-69af-447a-affb-2e2277d5cd78 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:41.535053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a934f2ac-69af-447a-affb-2e2277d5cd78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:41.536627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a934f2ac-69af-447a-affb-2e2277d5cd78 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:41.585319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:41.628647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934f2ac-69af-447a-affb-2e2277d5cd78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:41.630314 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a934f2ac-69af-447a-affb-2e2277d5cd78 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:41.648229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 724/1428] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:42.711779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65d92d-577b-4d9d-9244-461c31558e53 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:42.712364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65d92d-577b-4d9d-9244-461c31558e53 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:42.712364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c65d92d-577b-4d9d-9244-461c31558e53 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:42.713757 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 705/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:36:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:42.720811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ea932-91e4-42d1-a659-afc38c97edfa None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:42.721390 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ea932-91e4-42d1-a659-afc38c97edfa None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:42.721551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ea932-91e4-42d1-a659-afc38c97edfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:42.800942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-052ea932-91e4-42d1-a659-afc38c97edfa None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:42.802137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-052ea932-91e4-42d1-a659-afc38c97edfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:42.896746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 725/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:36:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:43.051919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65d92d-577b-4d9d-9244-461c31558e53 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:43.053481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2c65d92d-577b-4d9d-9244-461c31558e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:43.107257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:43.207325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c65d92d-577b-4d9d-9244-461c31558e53 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:43.325575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67299194-a268-4564-a6f0-416db36eeb84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:43.326181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67299194-a268-4564-a6f0-416db36eeb84 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:43.326451 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67299194-a268-4564-a6f0-416db36eeb84 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:43.326715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67299194-a268-4564-a6f0-416db36eeb84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:43.328309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-67299194-a268-4564-a6f0-416db36eeb84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:43.336146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:43.418599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67299194-a268-4564-a6f0-416db36eeb84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:43.420193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67299194-a268-4564-a6f0-416db36eeb84 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:43.439661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 706/1431] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:44.498178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ff520-9a87-4e93-9343-160aa91e0b12 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:44.498809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ff520-9a87-4e93-9343-160aa91e0b12 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:44.498809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ff520-9a87-4e93-9343-160aa91e0b12 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:44.500039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 726/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:44.506985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c1bb8-43ef-4735-bb84-3f1eebef177b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:44.507471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c1bb8-43ef-4735-bb84-3f1eebef177b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:44.507584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6c1bb8-43ef-4735-bb84-3f1eebef177b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:44.584109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2a6c1bb8-43ef-4735-bb84-3f1eebef177b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:44.585406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2a6c1bb8-43ef-4735-bb84-3f1eebef177b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:44.651443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:44.685973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 707/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:36:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:44.847479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ff520-9a87-4e93-9343-160aa91e0b12 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:44.849087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-915ff520-9a87-4e93-9343-160aa91e0b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:44.866723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:44.995864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ff520-9a87-4e93-9343-160aa91e0b12 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:45.082605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:45.110169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406339d-07db-463f-ab0d-025eb1fb5b31 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:45.110743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406339d-07db-463f-ab0d-025eb1fb5b31 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:45.111129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406339d-07db-463f-ab0d-025eb1fb5b31 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:45.111495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406339d-07db-463f-ab0d-025eb1fb5b31 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:45.114141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2406339d-07db-463f-ab0d-025eb1fb5b31 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:45.205604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406339d-07db-463f-ab0d-025eb1fb5b31 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:45.207190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406339d-07db-463f-ab0d-025eb1fb5b31 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:45.225400 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 727/1434] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:46.284026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e12d8-42b5-487a-a6ab-058ce807402a None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:46.284606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e12d8-42b5-487a-a6ab-058ce807402a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:46.284606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509e12d8-42b5-487a-a6ab-058ce807402a None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:46.286071 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 708/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:36:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:46.293114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70724b52-d675-4418-a0df-15c8f5119a0b None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:46.293635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70724b52-d675-4418-a0df-15c8f5119a0b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:46.293635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70724b52-d675-4418-a0df-15c8f5119a0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:46.373305 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-70724b52-d675-4418-a0df-15c8f5119a0b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:46.374527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-70724b52-d675-4418-a0df-15c8f5119a0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:46.418306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:46.478989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 728/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:36:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:46.631185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e12d8-42b5-487a-a6ab-058ce807402a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:46.632695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-509e12d8-42b5-487a-a6ab-058ce807402a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:46.786210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509e12d8-42b5-487a-a6ab-058ce807402a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:46.851739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:46.911930 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ae92cb-faa8-4aaf-a005-39f058420b8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:46.912546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae92cb-faa8-4aaf-a005-39f058420b8c service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:46.912852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae92cb-faa8-4aaf-a005-39f058420b8c service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:46.913116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ae92cb-faa8-4aaf-a005-39f058420b8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:46.914694 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e1ae92cb-faa8-4aaf-a005-39f058420b8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:47.004212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae92cb-faa8-4aaf-a005-39f058420b8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:47.006340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ae92cb-faa8-4aaf-a005-39f058420b8c service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:47.024884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 709/1437] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:48.068283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f3a7f-256c-405b-be9f-05269841916e None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:48.068894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f3a7f-256c-405b-be9f-05269841916e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:48.068894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230f3a7f-256c-405b-be9f-05269841916e None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:48.070204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 729/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:48.076254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159a5da-02c2-486b-9418-5b2c9c665330 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:48.076730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159a5da-02c2-486b-9418-5b2c9c665330 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:48.076859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4159a5da-02c2-486b-9418-5b2c9c665330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:48.154729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4159a5da-02c2-486b-9418-5b2c9c665330 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:48.155935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4159a5da-02c2-486b-9418-5b2c9c665330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:48.158097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:48.253960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 710/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:36:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:48.403708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f3a7f-256c-405b-be9f-05269841916e None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:48.405281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-230f3a7f-256c-405b-be9f-05269841916e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:48.551979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230f3a7f-256c-405b-be9f-05269841916e None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:48.579566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:48.666195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693655ae-8688-4430-9d6b-c29b287bfa93 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:48.666710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693655ae-8688-4430-9d6b-c29b287bfa93 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:48.666982 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693655ae-8688-4430-9d6b-c29b287bfa93 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:48.667246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693655ae-8688-4430-9d6b-c29b287bfa93 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:48.668901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-693655ae-8688-4430-9d6b-c29b287bfa93 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:48.768147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693655ae-8688-4430-9d6b-c29b287bfa93 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:48.769764 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693655ae-8688-4430-9d6b-c29b287bfa93 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:48.790156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 730/1440] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:50.892614 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7ece7-ce37-4ad0-80ad-5d9e8202d2b9 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:50.893208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7ece7-ce37-4ad0-80ad-5d9e8202d2b9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:50.893208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd7ece7-ce37-4ad0-80ad-5d9e8202d2b9 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:50.894533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 711/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:36:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:50.899814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12dbba-3771-4df3-b795-5119a7d9cf8c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:50.900246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12dbba-3771-4df3-b795-5119a7d9cf8c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:50.900383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a12dbba-3771-4df3-b795-5119a7d9cf8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:50.940738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:50.966762 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7a12dbba-3771-4df3-b795-5119a7d9cf8c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:50.967986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7a12dbba-3771-4df3-b795-5119a7d9cf8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:51.064521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 731/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:36:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:51.349312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7ece7-ce37-4ad0-80ad-5d9e8202d2b9 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:51.351038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1bd7ece7-ce37-4ad0-80ad-5d9e8202d2b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:51.453632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd7ece7-ce37-4ad0-80ad-5d9e8202d2b9 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:51.485610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:51.527037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72406213-846f-4a2b-93f5-1a3edf2e51e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:51.527640 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72406213-846f-4a2b-93f5-1a3edf2e51e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:51.527913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72406213-846f-4a2b-93f5-1a3edf2e51e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:51.528179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72406213-846f-4a2b-93f5-1a3edf2e51e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:51.529802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-72406213-846f-4a2b-93f5-1a3edf2e51e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:51.638162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72406213-846f-4a2b-93f5-1a3edf2e51e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:51.639585 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72406213-846f-4a2b-93f5-1a3edf2e51e5 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:51.658200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 712/1443] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:54.146749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c0598-5128-4d6e-b680-803cccfd23a1 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:54.147352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c0598-5128-4d6e-b680-803cccfd23a1 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:54.147352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c0598-5128-4d6e-b680-803cccfd23a1 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:54.148608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 732/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:36:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:54.153751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e24dc-492d-4ae1-9f4c-24266138f2a5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:54.154199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e24dc-492d-4ae1-9f4c-24266138f2a5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:54.154297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e24dc-492d-4ae1-9f4c-24266138f2a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:54.221904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ad3e24dc-492d-4ae1-9f4c-24266138f2a5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:54.223111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ad3e24dc-492d-4ae1-9f4c-24266138f2a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:54.236728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:54.322841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 713/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:36:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:54.606715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c0598-5128-4d6e-b680-803cccfd23a1 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:54.608292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-696c0598-5128-4d6e-b680-803cccfd23a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:54.714845 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c0598-5128-4d6e-b680-803cccfd23a1 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:54.727191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:54.798163 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7475745f-8e83-429c-a770-fc4263b5b152 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:54.798766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475745f-8e83-429c-a770-fc4263b5b152 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:54.799042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475745f-8e83-429c-a770-fc4263b5b152 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:54.799311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7475745f-8e83-429c-a770-fc4263b5b152 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:54.800937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7475745f-8e83-429c-a770-fc4263b5b152 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:54.909264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475745f-8e83-429c-a770-fc4263b5b152 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:54.910697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7475745f-8e83-429c-a770-fc4263b5b152 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:54.928707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 733/1446] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:36:57.450617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a3ddd-4efe-4f93-9220-6c232c5045a8 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:57.451153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a3ddd-4efe-4f93-9220-6c232c5045a8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:57.451153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41a3ddd-4efe-4f93-9220-6c232c5045a8 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:57.452628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 714/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:36:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:36:57.460278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adda2e1-f04b-49b5-baab-8d2754ad0ded None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:57.460724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adda2e1-f04b-49b5-baab-8d2754ad0ded None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:57.460869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adda2e1-f04b-49b5-baab-8d2754ad0ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:57.508748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:57.527173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7adda2e1-f04b-49b5-baab-8d2754ad0ded None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:36:57.528366 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7adda2e1-f04b-49b5-baab-8d2754ad0ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:57.624645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 734/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:36:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:36:57.913309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a3ddd-4efe-4f93-9220-6c232c5045a8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:36:57.914889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f41a3ddd-4efe-4f93-9220-6c232c5045a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:58.019654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41a3ddd-4efe-4f93-9220-6c232c5045a8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:36:58.060479 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:36:58.095587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54daa4ea-f145-40f6-9c6c-62235e548607 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:36:58.096187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daa4ea-f145-40f6-9c6c-62235e548607 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:36:58.096458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daa4ea-f145-40f6-9c6c-62235e548607 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:36:58.096722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54daa4ea-f145-40f6-9c6c-62235e548607 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:36:58.098428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-54daa4ea-f145-40f6-9c6c-62235e548607 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:36:58.205568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54daa4ea-f145-40f6-9c6c-62235e548607 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:36:58.206915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54daa4ea-f145-40f6-9c6c-62235e548607 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:36:58.225936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 715/1449] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:36:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:00.481881 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8858a-f968-4f96-9533-aae19aea37d0 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:00.482464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8858a-f968-4f96-9533-aae19aea37d0 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:00.482464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c8858a-f968-4f96-9533-aae19aea37d0 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:00.483773 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 735/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:00.489203 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dd734-5e2e-4eb6-962b-4be180ca73fd None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:00.489768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dd734-5e2e-4eb6-962b-4be180ca73fd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:00.489768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937dd734-5e2e-4eb6-962b-4be180ca73fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:00.555840 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-937dd734-5e2e-4eb6-962b-4be180ca73fd None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:00.557070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-937dd734-5e2e-4eb6-962b-4be180ca73fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:00.577730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:00.653357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 716/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:37:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:00.954515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8858a-f968-4f96-9533-aae19aea37d0 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:00.956071 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-89c8858a-f968-4f96-9533-aae19aea37d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:01.061401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c8858a-f968-4f96-9533-aae19aea37d0 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:01.082479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:01.136201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3428875b-8a74-41c3-b31a-e8617e10e3c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:01.136809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428875b-8a74-41c3-b31a-e8617e10e3c5 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:01.137081 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428875b-8a74-41c3-b31a-e8617e10e3c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:01.137346 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3428875b-8a74-41c3-b31a-e8617e10e3c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:01.138922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3428875b-8a74-41c3-b31a-e8617e10e3c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:01.244459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428875b-8a74-41c3-b31a-e8617e10e3c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:01.245817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3428875b-8a74-41c3-b31a-e8617e10e3c5 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:01.263591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 736/1452] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:03.513572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965933a-2637-4224-9d5a-06952412e066 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:03.514147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965933a-2637-4224-9d5a-06952412e066 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:03.514147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3965933a-2637-4224-9d5a-06952412e066 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:03.515455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 717/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:37:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:03.521874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e9087-4a25-4022-b3c5-15bb2495a35c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:03.522290 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e9087-4a25-4022-b3c5-15bb2495a35c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:03.522426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e9087-4a25-4022-b3c5-15bb2495a35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:03.591182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:03.594040 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-5e2e9087-4a25-4022-b3c5-15bb2495a35c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:03.595789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5e2e9087-4a25-4022-b3c5-15bb2495a35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:03.693919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 737/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:37:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:37:03.978130 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965933a-2637-4224-9d5a-06952412e066 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:03.979666 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3965933a-2637-4224-9d5a-06952412e066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:04.082228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3965933a-2637-4224-9d5a-06952412e066 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:04.135014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:04.160258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78189e1d-5133-4da5-a1fe-319ef37b3a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:04.161653 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78189e1d-5133-4da5-a1fe-319ef37b3a9c service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:04.161946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78189e1d-5133-4da5-a1fe-319ef37b3a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:04.162205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78189e1d-5133-4da5-a1fe-319ef37b3a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:04.163847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-78189e1d-5133-4da5-a1fe-319ef37b3a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:04.273959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78189e1d-5133-4da5-a1fe-319ef37b3a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:04.275306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78189e1d-5133-4da5-a1fe-319ef37b3a9c service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:04.293795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 718/1455] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:06.515571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dad72-d20d-42d1-83e9-73093eaa6424 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:06.516111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dad72-d20d-42d1-83e9-73093eaa6424 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:06.516111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1dad72-d20d-42d1-83e9-73093eaa6424 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:06.517741 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 738/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:37:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:06.523187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391c71-b88c-4009-bb0e-cdebf1d6b53d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:06.523583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391c71-b88c-4009-bb0e-cdebf1d6b53d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:06.523730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83391c71-b88c-4009-bb0e-cdebf1d6b53d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:06.590018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-83391c71-b88c-4009-bb0e-cdebf1d6b53d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:06.591221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-83391c71-b88c-4009-bb0e-cdebf1d6b53d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:06.620279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:06.687911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 719/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:37:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:06.974799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dad72-d20d-42d1-83e9-73093eaa6424 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:06.976367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8c1dad72-d20d-42d1-83e9-73093eaa6424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:07.200292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1dad72-d20d-42d1-83e9-73093eaa6424 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:07.227865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:07.273506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e3188-0e5b-4485-ac82-27f41b47bf7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:07.274095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e3188-0e5b-4485-ac82-27f41b47bf7b service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:07.274369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e3188-0e5b-4485-ac82-27f41b47bf7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:07.274640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e3188-0e5b-4485-ac82-27f41b47bf7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:07.276203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c77e3188-0e5b-4485-ac82-27f41b47bf7b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:07.381769 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e3188-0e5b-4485-ac82-27f41b47bf7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:07.383184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77e3188-0e5b-4485-ac82-27f41b47bf7b service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:07.401365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 739/1458] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:09.592727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b3357-43af-4881-9432-c41551969575 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:09.593181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b3357-43af-4881-9432-c41551969575 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:09.593334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03b3357-43af-4881-9432-c41551969575 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:09.594686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 720/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:09.600440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979ed61-030a-4c16-995a-63d425dbffda None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:09.600917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979ed61-030a-4c16-995a-63d425dbffda None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:09.601011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5979ed61-030a-4c16-995a-63d425dbffda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:09.665501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-5979ed61-030a-4c16-995a-63d425dbffda None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:09.666694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5979ed61-030a-4c16-995a-63d425dbffda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:09.674766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:09.762382 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 740/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:37:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:10.049812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b3357-43af-4881-9432-c41551969575 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:10.051532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f03b3357-43af-4881-9432-c41551969575 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:10.155995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03b3357-43af-4881-9432-c41551969575 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:10.210619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:10.228812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d74b0c-2b9e-4b4c-a2aa-866627911bf0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:10.229412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d74b0c-2b9e-4b4c-a2aa-866627911bf0 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:10.229685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d74b0c-2b9e-4b4c-a2aa-866627911bf0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:10.229953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d74b0c-2b9e-4b4c-a2aa-866627911bf0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:10.231550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-26d74b0c-2b9e-4b4c-a2aa-866627911bf0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:10.341378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d74b0c-2b9e-4b4c-a2aa-866627911bf0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:10.342710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d74b0c-2b9e-4b4c-a2aa-866627911bf0 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:10.360383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 721/1461] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:12.579970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012759bb-386d-41b7-922d-c225364ec76c None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:12.580395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012759bb-386d-41b7-922d-c225364ec76c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:12.580501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-012759bb-386d-41b7-922d-c225364ec76c None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:12.582281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 741/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:37:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:12.588306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5571ea-45ef-4a18-af5c-3fc7c9568136 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:12.588716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5571ea-45ef-4a18-af5c-3fc7c9568136 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:12.588880 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5571ea-45ef-4a18-af5c-3fc7c9568136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:12.655860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-aa5571ea-45ef-4a18-af5c-3fc7c9568136 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:12.657076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-aa5571ea-45ef-4a18-af5c-3fc7c9568136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:12.695815 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:12.755205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 722/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:37:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:13.041933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012759bb-386d-41b7-922d-c225364ec76c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:13.043495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-012759bb-386d-41b7-922d-c225364ec76c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:13.154925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-012759bb-386d-41b7-922d-c225364ec76c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:13.191330 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:13.226741 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d3a64-9002-4a21-8eb8-83b40cfb5d67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:13.227336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d3a64-9002-4a21-8eb8-83b40cfb5d67 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:13.227609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d3a64-9002-4a21-8eb8-83b40cfb5d67 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:13.227871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d3a64-9002-4a21-8eb8-83b40cfb5d67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:13.229652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e53d3a64-9002-4a21-8eb8-83b40cfb5d67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:13.341089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d3a64-9002-4a21-8eb8-83b40cfb5d67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:13.342552 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d3a64-9002-4a21-8eb8-83b40cfb5d67 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:13.361156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 742/1464] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:15.531773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fb519-7e56-4369-8e95-ff1108af7c57 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:15.532399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fb519-7e56-4369-8e95-ff1108af7c57 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:15.532399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035fb519-7e56-4369-8e95-ff1108af7c57 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:15.533682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 723/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:37:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:15.539541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7224d33-d372-49bd-be97-b4a991030c1a None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:15.539965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7224d33-d372-49bd-be97-b4a991030c1a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:15.540064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7224d33-d372-49bd-be97-b4a991030c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:15.606507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c7224d33-d372-49bd-be97-b4a991030c1a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:15.607699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c7224d33-d372-49bd-be97-b4a991030c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:15.625218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:15.702933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 743/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:37:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:15.994106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fb519-7e56-4369-8e95-ff1108af7c57 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:15.995643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-035fb519-7e56-4369-8e95-ff1108af7c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:16.099338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035fb519-7e56-4369-8e95-ff1108af7c57 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:16.166447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:16.173748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae878e6-0af7-4a0a-8982-c7e3c223e2a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:16.174352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae878e6-0af7-4a0a-8982-c7e3c223e2a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:16.174625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae878e6-0af7-4a0a-8982-c7e3c223e2a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:16.174890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae878e6-0af7-4a0a-8982-c7e3c223e2a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:16.177633 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cae878e6-0af7-4a0a-8982-c7e3c223e2a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:16.286476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae878e6-0af7-4a0a-8982-c7e3c223e2a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:16.287865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae878e6-0af7-4a0a-8982-c7e3c223e2a4 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:16.307593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 724/1467] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:37:18.483800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261ba9f-ccde-4a92-9ed1-9747f206fb95 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:18.484367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261ba9f-ccde-4a92-9ed1-9747f206fb95 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:18.484367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6261ba9f-ccde-4a92-9ed1-9747f206fb95 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:18.485769 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 744/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:37:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:18.491704 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826f4f6-23f0-46fe-8955-eb08cc17fd08 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:18.492147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826f4f6-23f0-46fe-8955-eb08cc17fd08 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:18.492260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9826f4f6-23f0-46fe-8955-eb08cc17fd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:18.557925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-9826f4f6-23f0-46fe-8955-eb08cc17fd08 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:18.559119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9826f4f6-23f0-46fe-8955-eb08cc17fd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:18.607813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:18.657296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 725/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:37:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:18.959331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261ba9f-ccde-4a92-9ed1-9747f206fb95 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:18.960917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6261ba9f-ccde-4a92-9ed1-9747f206fb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:19.063881 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6261ba9f-ccde-4a92-9ed1-9747f206fb95 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:19.111260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:19.137471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff6af71-0688-44d8-adb9-3e4be0f1b4da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:19.138066 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6af71-0688-44d8-adb9-3e4be0f1b4da service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:19.138339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6af71-0688-44d8-adb9-3e4be0f1b4da service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:19.138604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff6af71-0688-44d8-adb9-3e4be0f1b4da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:19.140188 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6ff6af71-0688-44d8-adb9-3e4be0f1b4da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:19.252575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6af71-0688-44d8-adb9-3e4be0f1b4da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:19.254566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff6af71-0688-44d8-adb9-3e4be0f1b4da service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:19.272591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 745/1470] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:28.655145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=22 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:29.270418 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=66 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:39.158441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582ad47-0a95-4693-8a0f-8db850e00bea None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:39.159008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582ad47-0a95-4693-8a0f-8db850e00bea None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:39.159008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7582ad47-0a95-4693-8a0f-8db850e00bea None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:39.160408 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 726/1471] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:37:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:39.165606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b715199-a99d-4e11-8d80-f1ed85b6b58f None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:39.166008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b715199-a99d-4e11-8d80-f1ed85b6b58f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:39.166161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b715199-a99d-4e11-8d80-f1ed85b6b58f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:39.232075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-9b715199-a99d-4e11-8d80-f1ed85b6b58f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:39.233226 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9b715199-a99d-4e11-8d80-f1ed85b6b58f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:39.329676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 746/1472] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:37:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:39.627086 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3516a4-5994-43be-8cea-cf9bd6f6f8da None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:39.627370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3516a4-5994-43be-8cea-cf9bd6f6f8da None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:39.627643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3516a4-5994-43be-8cea-cf9bd6f6f8da None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:39.630081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 727/1473] 10.222.0.39 () {54 vars in 961 bytes} [Tue Sep 6 22:37:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:39.634976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ccca0-8309-4c7d-a17d-35d0afb4bb60 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:39.635308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ccca0-8309-4c7d-a17d-35d0afb4bb60 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:39.635576 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0ccca0-8309-4c7d-a17d-35d0afb4bb60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:39.710026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-6c0ccca0-8309-4c7d-a17d-35d0afb4bb60 None None] MFA Rules not processed for user `42cf265450454ff9a27a29093336205e`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:39.711214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6c0ccca0-8309-4c7d-a17d-35d0afb4bb60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:39.731072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:39.786772 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 747/1474] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Sep 6 22:37:39 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:39.794645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3516a4-5994-43be-8cea-cf9bd6f6f8da None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:39.796158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2a3516a4-5994-43be-8cea-cf9bd6f6f8da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:39.899539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3516a4-5994-43be-8cea-cf9bd6f6f8da None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:39.970570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6da301-361f-4185-b2bd-b8cab59829ba service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42cf265450454ff9a27a29093336205e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:39.971165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6da301-361f-4185-b2bd-b8cab59829ba service glance] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:39.971439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6da301-361f-4185-b2bd-b8cab59829ba service glance] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:39.971709 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6da301-361f-4185-b2bd-b8cab59829ba service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:39.973323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6c6da301-361f-4185-b2bd-b8cab59829ba service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:40.026710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:40.080986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6da301-361f-4185-b2bd-b8cab59829ba service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:40.083304 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6da301-361f-4185-b2bd-b8cab59829ba service glance] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:40.102104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 728/1475] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:42.320406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6fc11-6818-4229-8e2a-ea02bbb66091 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:42.320967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6fc11-6818-4229-8e2a-ea02bbb66091 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:42.320967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba6fc11-6818-4229-8e2a-ea02bbb66091 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:42.322276 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 748/1476] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:37:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:42.327600 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f015f2-e3a4-457c-8654-6a9a5c3f749f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:42.327996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f015f2-e3a4-457c-8654-6a9a5c3f749f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:42.328169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f015f2-e3a4-457c-8654-6a9a5c3f749f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:42.397470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-05f015f2-e3a4-457c-8654-6a9a5c3f749f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:42.398674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-05f015f2-e3a4-457c-8654-6a9a5c3f749f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:42.492825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 729/1477] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:37:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:37:42.762698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6fc11-6818-4229-8e2a-ea02bbb66091 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:42.764231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-bba6fc11-6818-4229-8e2a-ea02bbb66091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:42.889477 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba6fc11-6818-4229-8e2a-ea02bbb66091 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:42.937949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:42.985678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510914d7-fedb-4229-b373-1ed95506259c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:42.986287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510914d7-fedb-4229-b373-1ed95506259c admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:42.986561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510914d7-fedb-4229-b373-1ed95506259c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:42.986828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510914d7-fedb-4229-b373-1ed95506259c admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:42.988072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510914d7-fedb-4229-b373-1ed95506259c admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:42.989936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: WARNING py.warnings [None req-510914d7-fedb-4229-b373-1ed95506259c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:37:42.989936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: warnings.warn(msg) Sep 06 22:37:42.989936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: Sep 06 22:37:42.991070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510914d7-fedb-4229-b373-1ed95506259c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:43.010787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 749/1478] 10.222.0.39 () {56 vars in 1108 bytes} [Tue Sep 6 22:37:42 2022] GET /identity/v3/projects => generated 2553 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:37:45.199376 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4540ae2-d630-4a68-93b0-98f34990cf3f None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:45.199976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4540ae2-d630-4a68-93b0-98f34990cf3f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:45.199976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4540ae2-d630-4a68-93b0-98f34990cf3f None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:45.201286 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 730/1479] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:37:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:45.207258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6c8f1-2d68-4319-93cf-3710dce7d7f8 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:45.207695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6c8f1-2d68-4319-93cf-3710dce7d7f8 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:45.207800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd6c8f1-2d68-4319-93cf-3710dce7d7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:45.274295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-9dd6c8f1-2d68-4319-93cf-3710dce7d7f8 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:45.275434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9dd6c8f1-2d68-4319-93cf-3710dce7d7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:45.371605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 750/1480] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:37:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:45.665559 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4540ae2-d630-4a68-93b0-98f34990cf3f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:45.667075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d4540ae2-d630-4a68-93b0-98f34990cf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:45.677475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:45.819945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4540ae2-d630-4a68-93b0-98f34990cf3f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:45.899101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:45.935384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95999d-6ff2-4745-8609-b0bb525f89ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:45.936239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95999d-6ff2-4745-8609-b0bb525f89ee service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:45.936535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95999d-6ff2-4745-8609-b0bb525f89ee service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:45.936827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95999d-6ff2-4745-8609-b0bb525f89ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:45.938965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ff95999d-6ff2-4745-8609-b0bb525f89ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:46.035385 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95999d-6ff2-4745-8609-b0bb525f89ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:46.036995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95999d-6ff2-4745-8609-b0bb525f89ee service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:46.056293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 731/1481] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:48.446253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237915c-df42-4b3b-9d5d-8fc6a157faaf None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:48.446818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237915c-df42-4b3b-9d5d-8fc6a157faaf None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:48.446818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5237915c-df42-4b3b-9d5d-8fc6a157faaf None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:48.448161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 751/1482] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:48.453585 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1b086-c211-43a9-a96f-0c915b91d274 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:48.453974 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1b086-c211-43a9-a96f-0c915b91d274 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:48.454136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1b086-c211-43a9-a96f-0c915b91d274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:48.526209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a5d1b086-c211-43a9-a96f-0c915b91d274 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:48.527735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a5d1b086-c211-43a9-a96f-0c915b91d274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:48.574340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:48.625496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 732/1483] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:37:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:48.908668 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237915c-df42-4b3b-9d5d-8fc6a157faaf None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:48.911173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5237915c-df42-4b3b-9d5d-8fc6a157faaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:48.935079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:49.016592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5237915c-df42-4b3b-9d5d-8fc6a157faaf None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:49.095895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794a5793-1f8b-4e0b-abea-75e5c0370343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:49.096501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a5793-1f8b-4e0b-abea-75e5c0370343 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:49.097408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a5793-1f8b-4e0b-abea-75e5c0370343 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:49.097679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794a5793-1f8b-4e0b-abea-75e5c0370343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:49.099286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-794a5793-1f8b-4e0b-abea-75e5c0370343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:49.171714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:49.207924 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a5793-1f8b-4e0b-abea-75e5c0370343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:49.209387 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794a5793-1f8b-4e0b-abea-75e5c0370343 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:49.227799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 752/1484] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:51.485662 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6132b5-8ded-4f51-9028-bccce572177b None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:51.486236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6132b5-8ded-4f51-9028-bccce572177b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:51.486236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6132b5-8ded-4f51-9028-bccce572177b None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:51.488406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 733/1485] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:37:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:51.493850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca260f3c-3f6b-47b5-a592-46325a01b942 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:51.494290 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca260f3c-3f6b-47b5-a592-46325a01b942 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:51.494396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca260f3c-3f6b-47b5-a592-46325a01b942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:51.560172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-ca260f3c-3f6b-47b5-a592-46325a01b942 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:51.561413 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ca260f3c-3f6b-47b5-a592-46325a01b942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:51.658119 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 753/1486] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:37:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:51.954838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6132b5-8ded-4f51-9028-bccce572177b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:51.956903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cf6132b5-8ded-4f51-9028-bccce572177b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:52.059162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6132b5-8ded-4f51-9028-bccce572177b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:52.129108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:52.130839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e84b99-4e60-4811-9473-249cec3590a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:52.131471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e84b99-4e60-4811-9473-249cec3590a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:52.131945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e84b99-4e60-4811-9473-249cec3590a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:52.132438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e84b99-4e60-4811-9473-249cec3590a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:52.134307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-84e84b99-4e60-4811-9473-249cec3590a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:52.241962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e84b99-4e60-4811-9473-249cec3590a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:52.243318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e84b99-4e60-4811-9473-249cec3590a8 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:52.261395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 734/1487] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:54.688915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ccb90-1b72-434e-8616-e60120ea81d6 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:54.689473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ccb90-1b72-434e-8616-e60120ea81d6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:54.689473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68ccb90-1b72-434e-8616-e60120ea81d6 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:54.691360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 754/1488] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:37:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:37:54.696864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e66ce-c548-410d-bc23-e3795b410c12 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:54.697293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e66ce-c548-410d-bc23-e3795b410c12 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:54.697406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e66ce-c548-410d-bc23-e3795b410c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:54.762846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-298e66ce-c548-410d-bc23-e3795b410c12 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:54.764035 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-298e66ce-c548-410d-bc23-e3795b410c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:54.817055 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:54.859939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 735/1489] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:37:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:37:55.154968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ccb90-1b72-434e-8616-e60120ea81d6 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:55.156593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d68ccb90-1b72-434e-8616-e60120ea81d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:55.192549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:55.259826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68ccb90-1b72-434e-8616-e60120ea81d6 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:55.333158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40917309-83e4-4fae-a104-0d09f801184a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:55.333753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40917309-83e4-4fae-a104-0d09f801184a service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:55.334026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40917309-83e4-4fae-a104-0d09f801184a service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:55.334293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40917309-83e4-4fae-a104-0d09f801184a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:55.335881 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-40917309-83e4-4fae-a104-0d09f801184a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:55.415305 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:55.445035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40917309-83e4-4fae-a104-0d09f801184a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:55.446448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40917309-83e4-4fae-a104-0d09f801184a service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:55.465003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 755/1490] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:37:57.814291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf58e91-db68-4936-95bd-a27fedb7caba None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:57.814852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf58e91-db68-4936-95bd-a27fedb7caba None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:57.814852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf58e91-db68-4936-95bd-a27fedb7caba None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:57.816737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 736/1491] 10.222.0.39 () {54 vars in 927 bytes} [Tue Sep 6 22:37:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:37:57.821956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5112fa-c58c-41fe-8d72-948cda5f8eb4 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:57.822418 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5112fa-c58c-41fe-8d72-948cda5f8eb4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:57.822529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5112fa-c58c-41fe-8d72-948cda5f8eb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:57.889253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-8d5112fa-c58c-41fe-8d72-948cda5f8eb4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:37:57.890465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8d5112fa-c58c-41fe-8d72-948cda5f8eb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:57.986348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 756/1492] 10.222.0.39 () {58 vars in 1024 bytes} [Tue Sep 6 22:37:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:37:58.275675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf58e91-db68-4936-95bd-a27fedb7caba None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:37:58.277249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bcf58e91-db68-4936-95bd-a27fedb7caba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:58.377234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf58e91-db68-4936-95bd-a27fedb7caba None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:37:58.448890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ea17c-cf4d-4921-9133-4c0c898cd2a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:37:58.449484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ea17c-cf4d-4921-9133-4c0c898cd2a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:37:58.449755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ea17c-cf4d-4921-9133-4c0c898cd2a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:37:58.450019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ea17c-cf4d-4921-9133-4c0c898cd2a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:37:58.451602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-598ea17c-cf4d-4921-9133-4c0c898cd2a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:37:58.460605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:37:58.559825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ea17c-cf4d-4921-9133-4c0c898cd2a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:37:58.561220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ea17c-cf4d-4921-9133-4c0c898cd2a6 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:37:58.579228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 737/1493] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:37:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:38:07.984500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=82 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:38:08.577057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=50 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:11.139928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c33d-9666-4f9e-9ec8-f4c0322b2f2d None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:11.140462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c33d-9666-4f9e-9ec8-f4c0322b2f2d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:11.140462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f4c33d-9666-4f9e-9ec8-f4c0322b2f2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:11.214994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e1f4c33d-9666-4f9e-9ec8-f4c0322b2f2d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:39:11.216199 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e1f4c33d-9666-4f9e-9ec8-f4c0322b2f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:11.314709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 757/1494] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:39:11.322454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a055c0b-a06a-4d73-8daf-3ecfb1a8e9b4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:11.322938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a055c0b-a06a-4d73-8daf-3ecfb1a8e9b4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:11.323056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a055c0b-a06a-4d73-8daf-3ecfb1a8e9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:11.399951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0a055c0b-a06a-4d73-8daf-3ecfb1a8e9b4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:39:11.401176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0a055c0b-a06a-4d73-8daf-3ecfb1a8e9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:11.495562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 738/1495] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:39:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:39:11.500264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c33d-9666-4f9e-9ec8-f4c0322b2f2d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:39:11.501495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e1f4c33d-9666-4f9e-9ec8-f4c0322b2f2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:11.512958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f4c33d-9666-4f9e-9ec8-f4c0322b2f2d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:39:11.574000 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:11.609612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ac761-eada-4286-b7d1-0bd8856c6ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:39:11.610220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ac761-eada-4286-b7d1-0bd8856c6ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:11.610480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ac761-eada-4286-b7d1-0bd8856c6ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:11.610746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26ac761-eada-4286-b7d1-0bd8856c6ab6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:11.611376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ac761-eada-4286-b7d1-0bd8856c6ab6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:39:11.611674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ac761-eada-4286-b7d1-0bd8856c6ab6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:39:11.612658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: WARNING py.warnings [None req-f26ac761-eada-4286-b7d1-0bd8856c6ab6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:39:11.612658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: warnings.warn(msg) Sep 06 22:39:11.612658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: Sep 06 22:39:11.613555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26ac761-eada-4286-b7d1-0bd8856c6ab6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:39:11.624369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 758/1496] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:39:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:39:11.632625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a055c0b-a06a-4d73-8daf-3ecfb1a8e9b4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:39:11.642270 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a055c0b-a06a-4d73-8daf-3ecfb1a8e9b4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:39:11.702537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:11.736880 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2fb30-fe0f-43f2-86fc-c9550373895e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:39:11.737487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2fb30-fe0f-43f2-86fc-c9550373895e admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:11.737768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2fb30-fe0f-43f2-86fc-c9550373895e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:11.738159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e2fb30-fe0f-43f2-86fc-c9550373895e admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:11.738789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2fb30-fe0f-43f2-86fc-c9550373895e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:39:11.740525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-62e2fb30-fe0f-43f2-86fc-c9550373895e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:39:11.740525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 06 22:39:11.740525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: Sep 06 22:39:11.741743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e2fb30-fe0f-43f2-86fc-c9550373895e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:39:11.796308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 739/1497] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:39:11 2022] POST /identity/v3/projects => generated 379 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:39:11.801137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ac761-eada-4286-b7d1-0bd8856c6ab6 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:39:11.802700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f26ac761-eada-4286-b7d1-0bd8856c6ab6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:11.836938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26ac761-eada-4286-b7d1-0bd8856c6ab6 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:39:11.931842 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbac44-b053-4fbb-ae8d-7738f25bd5b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:39:11.932441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbac44-b053-4fbb-ae8d-7738f25bd5b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:11.932716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbac44-b053-4fbb-ae8d-7738f25bd5b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:11.933012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efbac44-b053-4fbb-ae8d-7738f25bd5b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:11.933620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbac44-b053-4fbb-ae8d-7738f25bd5b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:39:11.934607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: WARNING py.warnings [None req-2efbac44-b053-4fbb-ae8d-7738f25bd5b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:39:11.934607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: warnings.warn(msg) Sep 06 22:39:11.934607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: Sep 06 22:39:11.935798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efbac44-b053-4fbb-ae8d-7738f25bd5b7 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:39:11.978474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 759/1498] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:39:11 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:39:11.982605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2fb30-fe0f-43f2-86fc-c9550373895e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:39:11.984147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-62e2fb30-fe0f-43f2-86fc-c9550373895e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:12.046332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e2fb30-fe0f-43f2-86fc-c9550373895e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:39:12.129860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:12.140505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1ff50-2e78-422c-8614-e8d5c90f1eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:39:12.141983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1ff50-2e78-422c-8614-e8d5c90f1eeb admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:12.142250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1ff50-2e78-422c-8614-e8d5c90f1eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:12.142510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec1ff50-2e78-422c-8614-e8d5c90f1eeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:12.143136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1ff50-2e78-422c-8614-e8d5c90f1eeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:39:12.144162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-6ec1ff50-2e78-422c-8614-e8d5c90f1eeb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:39:12.144162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 06 22:39:12.144162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: Sep 06 22:39:12.145046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec1ff50-2e78-422c-8614-e8d5c90f1eeb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:39:12.157084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 740/1499] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:39:11 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:39:12.161076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbac44-b053-4fbb-ae8d-7738f25bd5b7 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:39:12.162603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2efbac44-b053-4fbb-ae8d-7738f25bd5b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:12.173865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efbac44-b053-4fbb-ae8d-7738f25bd5b7 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:39:12.198594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:12.280177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ac676e-54f6-4667-ada9-200809194547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:39:12.280798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac676e-54f6-4667-ada9-200809194547 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:12.281073 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac676e-54f6-4667-ada9-200809194547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:12.281411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ac676e-54f6-4667-ada9-200809194547 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:12.282240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac676e-54f6-4667-ada9-200809194547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:39:12.283238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: WARNING py.warnings [None req-10ac676e-54f6-4667-ada9-200809194547 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:39:12.283238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: warnings.warn(msg) Sep 06 22:39:12.283238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: Sep 06 22:39:12.284116 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10ac676e-54f6-4667-ada9-200809194547 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:39:12.296336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 760/1500] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:39:12 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:39:12.301159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1ff50-2e78-422c-8614-e8d5c90f1eeb admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:39:12.302670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6ec1ff50-2e78-422c-8614-e8d5c90f1eeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:12.336192 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec1ff50-2e78-422c-8614-e8d5c90f1eeb admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:39:12.430479 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4ed7fe-895a-4a24-ae61-d0598bd22ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:39:12.431322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ed7fe-895a-4a24-ae61-d0598bd22ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:12.431598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ed7fe-895a-4a24-ae61-d0598bd22ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:12.431871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4ed7fe-895a-4a24-ae61-d0598bd22ab0 admin admin] PATH_INFO: `/v3/projects/d85ada203ec64c6cb3fb2df477903259/users/ba3852756be64b13ae31ef38a99c238f/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:12.454264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ed7fe-895a-4a24-ae61-d0598bd22ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d85ada203ec64c6cb3fb2df477903259, user_id=ba3852756be64b13ae31ef38a99c238f, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:39:12.455284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-cb4ed7fe-895a-4a24-ae61-d0598bd22ab0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:39:12.455284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 06 22:39:12.455284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: Sep 06 22:39:12.457245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4ed7fe-895a-4a24-ae61-d0598bd22ab0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:39:12.482188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 741/1501] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:39:12 2022] PUT /identity/v3/projects/d85ada203ec64c6cb3fb2df477903259/users/ba3852756be64b13ae31ef38a99c238f/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:39:12.495065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8428e2-5a95-4ab9-8002-988e18d2774c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:12.495350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8428e2-5a95-4ab9-8002-988e18d2774c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:12.495624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8428e2-5a95-4ab9-8002-988e18d2774c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:12.551803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-8b8428e2-5a95-4ab9-8002-988e18d2774c None None] MFA Rules not processed for user `ba3852756be64b13ae31ef38a99c238f`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:39:12.553012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8b8428e2-5a95-4ab9-8002-988e18d2774c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:12.631664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 761/1502] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:39:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:39:12.645570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1640f0d-afa7-435c-b4f4-c232a371cc6c None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:12.645849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1640f0d-afa7-435c-b4f4-c232a371cc6c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:12.646117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1640f0d-afa7-435c-b4f4-c232a371cc6c None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:12.647439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 742/1503] 10.222.0.39 () {54 vars in 964 bytes} [Tue Sep 6 22:39:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:39:12.652062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfd24a-0e55-484f-9685-5d543ea7f74e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:12.652363 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfd24a-0e55-484f-9685-5d543ea7f74e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:12.652619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bfd24a-0e55-484f-9685-5d543ea7f74e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:12.685070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:12.731492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-14bfd24a-0e55-484f-9685-5d543ea7f74e None None] MFA Rules not processed for user `d2da9e23277a4de2a61b9727737ab8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:39:12.732710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-14bfd24a-0e55-484f-9685-5d543ea7f74e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:12.807795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 762/1504] 10.222.0.39 () {58 vars in 1061 bytes} [Tue Sep 6 22:39:12 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:39:12.815882 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1640f0d-afa7-435c-b4f4-c232a371cc6c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:39:12.817423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b1640f0d-afa7-435c-b4f4-c232a371cc6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:12.866849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:12.921198 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1640f0d-afa7-435c-b4f4-c232a371cc6c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:39:12.994478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b582e4-b279-43a2-96c7-0cae437405f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:39:12.995004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b582e4-b279-43a2-96c7-0cae437405f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:12.995282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b582e4-b279-43a2-96c7-0cae437405f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:12.995549 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b582e4-b279-43a2-96c7-0cae437405f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:12.997161 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-45b582e4-b279-43a2-96c7-0cae437405f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:13.096024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b582e4-b279-43a2-96c7-0cae437405f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:39:13.097469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b582e4-b279-43a2-96c7-0cae437405f0 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:39:13.100924 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:13.115964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 743/1505] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:39:12 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:39:13.162690 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a422b-6555-4a98-8768-6d0d8a7720bd None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:13.163123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a422b-6555-4a98-8768-6d0d8a7720bd None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:13.163243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013a422b-6555-4a98-8768-6d0d8a7720bd None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:13.164457 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 763/1506] 10.222.0.39 () {52 vars in 830 bytes} [Tue Sep 6 22:39:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:39:13.168339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b582e4-b279-43a2-96c7-0cae437405f0 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:39:13.169588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-45b582e4-b279-43a2-96c7-0cae437405f0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:13.268142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b582e4-b279-43a2-96c7-0cae437405f0 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:39:13.368721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:39:13.369472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:13.369643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:13.369910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] PATH_INFO: `/v3/users/ba3852756be64b13ae31ef38a99c238f` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:13.383202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba3852756be64b13ae31ef38a99c238f)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:39:13.384228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:39:13.384228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 06 22:39:13.384228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: Sep 06 22:39:13.385520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:39:13.396706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:13.454389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba3852756be64b13ae31ef38a99c238f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:39:13.454823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba3852756be64b13ae31ef38a99c238f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:39:13.463768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba3852756be64b13ae31ef38a99c238f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:39:13.477946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba3852756be64b13ae31ef38a99c238f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:39:13.500437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba3852756be64b13ae31ef38a99c238f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:39:13.506221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 744/1507] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:39:13 2022] DELETE /identity/v3/users/ba3852756be64b13ae31ef38a99c238f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:39:13.510678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35617ad2-1c75-40af-b11d-c512c998ad5c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:13.510960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35617ad2-1c75-40af-b11d-c512c998ad5c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:13.511231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35617ad2-1c75-40af-b11d-c512c998ad5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:13.569176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-35617ad2-1c75-40af-b11d-c512c998ad5c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:39:13.570300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-35617ad2-1c75-40af-b11d-c512c998ad5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:13.645322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:13.658520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 764/1508] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:39:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:39:13.668110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:39:13.669359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:13.768146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11db23b-9003-431c-94e3-4e65defab8be admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:39:13.821991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:13.842900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b10e89-8b63-4595-9856-37b96ba485b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:39:13.843508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b10e89-8b63-4595-9856-37b96ba485b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:13.843895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b10e89-8b63-4595-9856-37b96ba485b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:13.844177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b10e89-8b63-4595-9856-37b96ba485b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:13.845902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-65b10e89-8b63-4595-9856-37b96ba485b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:13.951155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b10e89-8b63-4595-9856-37b96ba485b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:39:13.952562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b10e89-8b63-4595-9856-37b96ba485b1 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:39:13.970750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 745/1509] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:39:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:39:14.315621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35617ad2-1c75-40af-b11d-c512c998ad5c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:39:14.316908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-35617ad2-1c75-40af-b11d-c512c998ad5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:39:14.329270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35617ad2-1c75-40af-b11d-c512c998ad5c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:39:14.426646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:39:14.427273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:39:14.427549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:39:14.427820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] PATH_INFO: `/v3/projects/d85ada203ec64c6cb3fb2df477903259` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:39:14.438022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=d85ada203ec64c6cb3fb2df477903259)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:39:14.439037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: WARNING py.warnings [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:39:14.439037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: warnings.warn(msg) Sep 06 22:39:14.439037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: Sep 06 22:39:14.440226 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:39:14.497667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd85ada203ec64c6cb3fb2df477903259', 'request_id': 'req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:39:14.507232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd85ada203ec64c6cb3fb2df477903259', 'request_id': 'req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:39:14.513212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd85ada203ec64c6cb3fb2df477903259', 'request_id': 'req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:39:14.545644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] The token cache is being invalidate because project d85ada203ec64c6cb3fb2df477903259 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:39:14.545928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:39:14.546350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-91fa5557-8ed6-40d1-806b-62a8b1c44bbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:39:14.548238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 765/1510] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:39:14 2022] DELETE /identity/v3/projects/d85ada203ec64c6cb3fb2df477903259 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:39:23.968422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=62 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:39:24.543683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:40:41.966842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba623a7-36fe-4f78-87a0-6146501a949a None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:41.967471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba623a7-36fe-4f78-87a0-6146501a949a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:41.967471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba623a7-36fe-4f78-87a0-6146501a949a None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:41.968717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 746/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:40:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:40:41.974192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8684555-81da-4a5c-92bd-a3f739a8f5c4 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:41.974643 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8684555-81da-4a5c-92bd-a3f739a8f5c4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:41.974747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8684555-81da-4a5c-92bd-a3f739a8f5c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:42.057976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-f8684555-81da-4a5c-92bd-a3f739a8f5c4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:40:42.059196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f8684555-81da-4a5c-92bd-a3f739a8f5c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:40:42.162179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 766/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:40:42.317182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba623a7-36fe-4f78-87a0-6146501a949a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:40:42.319310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dba623a7-36fe-4f78-87a0-6146501a949a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:40:42.443846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba623a7-36fe-4f78-87a0-6146501a949a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:40:42.449360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396ac225-04dc-4787-810f-1dc8ab245d65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:40:42.449960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ac225-04dc-4787-810f-1dc8ab245d65 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:42.450241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ac225-04dc-4787-810f-1dc8ab245d65 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:42.450513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396ac225-04dc-4787-810f-1dc8ab245d65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:42.459146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ac225-04dc-4787-810f-1dc8ab245d65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:40:42.460722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396ac225-04dc-4787-810f-1dc8ab245d65 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:40:42.470660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 747/1513] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:40:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:40:43.564100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d901fe-0c74-4ffe-b38a-e22df18de9ab None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:43.564749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d901fe-0c74-4ffe-b38a-e22df18de9ab None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:43.564749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d901fe-0c74-4ffe-b38a-e22df18de9ab None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:43.566073 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 767/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:40:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:40:43.572424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875ed23-e19f-4040-a0f1-a9e0bf30c789 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:43.572879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875ed23-e19f-4040-a0f1-a9e0bf30c789 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:43.573016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7875ed23-e19f-4040-a0f1-a9e0bf30c789 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:43.593444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7875ed23-e19f-4040-a0f1-a9e0bf30c789 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:40:43.595173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7875ed23-e19f-4040-a0f1-a9e0bf30c789 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:40:43.603318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 748/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:40:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:40:43.768215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d901fe-0c74-4ffe-b38a-e22df18de9ab None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:40:43.782623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d901fe-0c74-4ffe-b38a-e22df18de9ab None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:40:43.788019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d5ab13-ed7f-41b3-be0e-87c0037467cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:40:43.788610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5ab13-ed7f-41b3-be0e-87c0037467cf service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:43.788925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5ab13-ed7f-41b3-be0e-87c0037467cf service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:43.789196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d5ab13-ed7f-41b3-be0e-87c0037467cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:43.797176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5ab13-ed7f-41b3-be0e-87c0037467cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:40:43.799320 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d5ab13-ed7f-41b3-be0e-87c0037467cf service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:40:43.808578 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 768/1516] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:40:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:40:43.900481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0750cb-98cd-4a81-bbbf-4f84196a1f81 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:43.900897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0750cb-98cd-4a81-bbbf-4f84196a1f81 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:43.901232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0750cb-98cd-4a81-bbbf-4f84196a1f81 None None] PATH_INFO: `/v3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:43.902617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 749/1517] 10.222.0.49 () {54 vars in 925 bytes} [Tue Sep 6 22:40:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:40:43.909195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247139d-2d11-4a51-8c47-2a6ebee9bf32 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:43.909482 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247139d-2d11-4a51-8c47-2a6ebee9bf32 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:43.909752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6247139d-2d11-4a51-8c47-2a6ebee9bf32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:43.972153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-6247139d-2d11-4a51-8c47-2a6ebee9bf32 None None] MFA Rules not processed for user `1d7e1b7dfa32448a88e0c90f10cdb2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:40:43.973433 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6247139d-2d11-4a51-8c47-2a6ebee9bf32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:40:43.990491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:40:44.046093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 769/1518] 10.222.0.49 () {58 vars in 1015 bytes} [Tue Sep 6 22:40:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:40:47.564230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b900-47fa-4a9d-8f85-ef7f37c6a4fc None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:47.564740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b900-47fa-4a9d-8f85-ef7f37c6a4fc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:47.564867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7947b900-47fa-4a9d-8f85-ef7f37c6a4fc None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:47.566196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 750/1519] 10.222.0.39 () {54 vars in 961 bytes} [Tue Sep 6 22:40:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:40:47.571582 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d531233-f30b-4248-acaf-d497de5b286f None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:47.572065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d531233-f30b-4248-acaf-d497de5b286f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:47.572185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d531233-f30b-4248-acaf-d497de5b286f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:47.592664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-3d531233-f30b-4248-acaf-d497de5b286f None None] MFA Rules not processed for user `1d7e1b7dfa32448a88e0c90f10cdb2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:40:47.593894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3d531233-f30b-4248-acaf-d497de5b286f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:40:47.607535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 770/1520] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Sep 6 22:40:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:40:47.611004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947b900-47fa-4a9d-8f85-ef7f37c6a4fc None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:40:47.624258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7947b900-47fa-4a9d-8f85-ef7f37c6a4fc None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:40:47.630205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d5a954-2bf0-457b-afba-2787347a89bf service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:40:47.630811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5a954-2bf0-457b-afba-2787347a89bf service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:47.632004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5a954-2bf0-457b-afba-2787347a89bf service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:47.632282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d5a954-2bf0-457b-afba-2787347a89bf service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:47.642032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5a954-2bf0-457b-afba-2787347a89bf service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:40:47.644116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d5a954-2bf0-457b-afba-2787347a89bf service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:40:47.659546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 751/1521] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:40:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:40:55.305970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44502861-85bb-4729-a680-24e761515880 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:55.306567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44502861-85bb-4729-a680-24e761515880 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:55.306567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44502861-85bb-4729-a680-24e761515880 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:55.307848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 771/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:40:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:40:55.314843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e4449-b928-4b3b-9516-7f043a0df6a2 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:55.315284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e4449-b928-4b3b-9516-7f043a0df6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:55.315383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e4449-b928-4b3b-9516-7f043a0df6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:55.335590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-647e4449-b928-4b3b-9516-7f043a0df6a2 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:40:55.336696 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-647e4449-b928-4b3b-9516-7f043a0df6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:40:55.344783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 752/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:40:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:40:55.497173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44502861-85bb-4729-a680-24e761515880 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:40:55.512299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44502861-85bb-4729-a680-24e761515880 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:40:55.518306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a84e1-39a7-4c34-8ede-2ecc6c45d1b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:40:55.518889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a84e1-39a7-4c34-8ede-2ecc6c45d1b8 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:55.519161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a84e1-39a7-4c34-8ede-2ecc6c45d1b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:55.519428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a84e1-39a7-4c34-8ede-2ecc6c45d1b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:55.527231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a84e1-39a7-4c34-8ede-2ecc6c45d1b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:40:55.528813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a84e1-39a7-4c34-8ede-2ecc6c45d1b8 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:40:55.537938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 772/1524] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:40:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:40:56.638615 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769a043-3d1f-481c-a42f-828213ed2b6a None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:56.639108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769a043-3d1f-481c-a42f-828213ed2b6a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:56.639219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3769a043-3d1f-481c-a42f-828213ed2b6a None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:56.640502 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 753/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 6 22:40:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:40:56.646892 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f1800-63e3-42d5-b7d3-2f7f3234a2bb None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:56.647317 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f1800-63e3-42d5-b7d3-2f7f3234a2bb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:56.647417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890f1800-63e3-42d5-b7d3-2f7f3234a2bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:56.666829 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-890f1800-63e3-42d5-b7d3-2f7f3234a2bb None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:40:56.668024 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-890f1800-63e3-42d5-b7d3-2f7f3234a2bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:40:56.676891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 773/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 6 22:40:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:40:56.829838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769a043-3d1f-481c-a42f-828213ed2b6a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:40:56.843895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3769a043-3d1f-481c-a42f-828213ed2b6a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:40:56.849915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ccc604-38fd-446b-a297-09223d8a2367 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:40:56.850505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccc604-38fd-446b-a297-09223d8a2367 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:40:56.850787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccc604-38fd-446b-a297-09223d8a2367 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:40:56.851061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ccc604-38fd-446b-a297-09223d8a2367 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:40:56.858934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccc604-38fd-446b-a297-09223d8a2367 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:40:56.860558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ccc604-38fd-446b-a297-09223d8a2367 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:40:56.869749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 754/1527] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:40:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:41:00.960667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f9492-d400-4874-af27-55d1be0339a5 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:00.961269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f9492-d400-4874-af27-55d1be0339a5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:00.961377 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f9492-d400-4874-af27-55d1be0339a5 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:00.962697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 774/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:41:00.968047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12f6fd-cb7b-4f69-bf93-caa158a8d23e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:00.968429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12f6fd-cb7b-4f69-bf93-caa158a8d23e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:00.968594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12f6fd-cb7b-4f69-bf93-caa158a8d23e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:00.988913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-9b12f6fd-cb7b-4f69-bf93-caa158a8d23e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:41:00.990101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9b12f6fd-cb7b-4f69-bf93-caa158a8d23e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:41:00.998083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 755/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:41:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:41:01.288389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f9492-d400-4874-af27-55d1be0339a5 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:41:01.290123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-da0f9492-d400-4874-af27-55d1be0339a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:41:01.368573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f9492-d400-4874-af27-55d1be0339a5 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:41:01.372154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9876b4c-3307-4d17-9794-5787ca19a289 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:41:01.372751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9876b4c-3307-4d17-9794-5787ca19a289 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:01.373050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9876b4c-3307-4d17-9794-5787ca19a289 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:01.373317 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9876b4c-3307-4d17-9794-5787ca19a289 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:01.382909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9876b4c-3307-4d17-9794-5787ca19a289 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:41:01.384342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9876b4c-3307-4d17-9794-5787ca19a289 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:41:01.394736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 775/1530] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:41:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:41:03.601075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7979d1-f395-4201-a4be-dab58fcd5715 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:03.601583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7979d1-f395-4201-a4be-dab58fcd5715 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:03.601691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7979d1-f395-4201-a4be-dab58fcd5715 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:03.602964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 756/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:41:03.608223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e98ad4-8613-4bd5-9d81-808864674ab5 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:03.608662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e98ad4-8613-4bd5-9d81-808864674ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:03.608788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e98ad4-8613-4bd5-9d81-808864674ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:03.628379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e4e98ad4-8613-4bd5-9d81-808864674ab5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:41:03.629682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e4e98ad4-8613-4bd5-9d81-808864674ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:41:03.637407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 776/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:41:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:41:03.926679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7979d1-f395-4201-a4be-dab58fcd5715 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:41:03.937803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7979d1-f395-4201-a4be-dab58fcd5715 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:41:03.941865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a242cbf5-f6dc-476d-9779-2b2edbe8e2a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:41:03.942447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242cbf5-f6dc-476d-9779-2b2edbe8e2a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:03.942717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242cbf5-f6dc-476d-9779-2b2edbe8e2a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:03.942978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a242cbf5-f6dc-476d-9779-2b2edbe8e2a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:03.951627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242cbf5-f6dc-476d-9779-2b2edbe8e2a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:41:03.953059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a242cbf5-f6dc-476d-9779-2b2edbe8e2a0 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:41:03.962293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 757/1533] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:41:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:41:07.741782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56f4d2-fe8d-4826-ac3f-9d8630722b24 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:07.742359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56f4d2-fe8d-4826-ac3f-9d8630722b24 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:07.742359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b56f4d2-fe8d-4826-ac3f-9d8630722b24 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:07.743685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 777/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:41:07.749053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee08fc-9691-4ff9-8be8-4f4c9f7b359d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:07.749466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee08fc-9691-4ff9-8be8-4f4c9f7b359d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:07.749665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee08fc-9691-4ff9-8be8-4f4c9f7b359d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:07.758415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=81 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:41:07.771141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1aee08fc-9691-4ff9-8be8-4f4c9f7b359d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:41:07.772331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1aee08fc-9691-4ff9-8be8-4f4c9f7b359d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:41:07.780012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 758/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:41:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:41:08.095952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56f4d2-fe8d-4826-ac3f-9d8630722b24 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:41:08.097738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8b56f4d2-fe8d-4826-ac3f-9d8630722b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:41:08.163409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:41:08.177144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b56f4d2-fe8d-4826-ac3f-9d8630722b24 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:41:08.180680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0233f8d5-8613-451d-a8e5-ed5ed0d3a4d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:41:08.181308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233f8d5-8613-451d-a8e5-ed5ed0d3a4d4 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:08.181584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233f8d5-8613-451d-a8e5-ed5ed0d3a4d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:08.181853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0233f8d5-8613-451d-a8e5-ed5ed0d3a4d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:08.191126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233f8d5-8613-451d-a8e5-ed5ed0d3a4d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:41:08.192532 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0233f8d5-8613-451d-a8e5-ed5ed0d3a4d4 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:41:08.201762 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 778/1536] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:41:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:41:10.448625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ad991-2d20-4594-a117-98d306f29497 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:10.449230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ad991-2d20-4594-a117-98d306f29497 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:10.449230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b76ad991-2d20-4594-a117-98d306f29497 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:10.450532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 759/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 6 22:41:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:41:10.456486 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46938c-cb76-419a-b9db-37d116ad1561 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:10.456941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46938c-cb76-419a-b9db-37d116ad1561 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:10.457086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46938c-cb76-419a-b9db-37d116ad1561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:10.476430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-fd46938c-cb76-419a-b9db-37d116ad1561 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:41:10.477672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-fd46938c-cb76-419a-b9db-37d116ad1561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:41:10.485548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 779/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 6 22:41:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:41:10.786547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ad991-2d20-4594-a117-98d306f29497 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:41:10.788285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b76ad991-2d20-4594-a117-98d306f29497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:41:10.801548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b76ad991-2d20-4594-a117-98d306f29497 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:41:10.805549 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319d7585-1059-4b06-b40c-794a74033377 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:41:10.806189 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d7585-1059-4b06-b40c-794a74033377 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:41:10.806465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d7585-1059-4b06-b40c-794a74033377 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:41:10.806729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319d7585-1059-4b06-b40c-794a74033377 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:41:10.816681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d7585-1059-4b06-b40c-794a74033377 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:41:10.818129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319d7585-1059-4b06-b40c-794a74033377 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:41:10.827409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 760/1539] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:41:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:41:20.471623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=11 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:41:20.823316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=8 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:34.319727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f5b93-5859-4e2e-8707-bd07339af380 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.321539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f5b93-5859-4e2e-8707-bd07339af380 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.321539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f5b93-5859-4e2e-8707-bd07339af380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.350894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-420f5b93-5859-4e2e-8707-bd07339af380 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:34.352671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-420f5b93-5859-4e2e-8707-bd07339af380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:34.366026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 780/1540] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:34.374964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c877-f434-4292-b0fb-78e1c6a1285b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.376556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c877-f434-4292-b0fb-78e1c6a1285b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.377081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5c877-f434-4292-b0fb-78e1c6a1285b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.406000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-04a5c877-f434-4292-b0fb-78e1c6a1285b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:34.408325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-04a5c877-f434-4292-b0fb-78e1c6a1285b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:34.422060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 761/1541] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:34.427639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f5b93-5859-4e2e-8707-bd07339af380 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:34.439232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f5b93-5859-4e2e-8707-bd07339af380 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:34.449185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f21b15-ad6d-4152-b0d2-bc5ba1b6f80f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:34.450342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f21b15-ad6d-4152-b0d2-bc5ba1b6f80f admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.450852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f21b15-ad6d-4152-b0d2-bc5ba1b6f80f admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.451347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f21b15-ad6d-4152-b0d2-bc5ba1b6f80f admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.452205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f21b15-ad6d-4152-b0d2-bc5ba1b6f80f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:42:34.452733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f21b15-ad6d-4152-b0d2-bc5ba1b6f80f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:34.455132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f21b15-ad6d-4152-b0d2-bc5ba1b6f80f admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:34.475755 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 781/1542] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:42:34.484736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5c877-f434-4292-b0fb-78e1c6a1285b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:34.504821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5c877-f434-4292-b0fb-78e1c6a1285b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:34.513918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678349ef-05c5-4327-9d77-e812de335fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:34.514557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678349ef-05c5-4327-9d77-e812de335fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.514836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678349ef-05c5-4327-9d77-e812de335fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.515104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678349ef-05c5-4327-9d77-e812de335fb3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.515751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678349ef-05c5-4327-9d77-e812de335fb3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:34.518808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678349ef-05c5-4327-9d77-e812de335fb3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:34.608737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 762/1543] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:34 2022] POST /identity/v3/projects => generated 403 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:34.615065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f21b15-ad6d-4152-b0d2-bc5ba1b6f80f admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:34.630621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f21b15-ad6d-4152-b0d2-bc5ba1b6f80f admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:34.723629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb873e-9b61-49bd-8a45-a36547485f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:34.724653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb873e-9b61-49bd-8a45-a36547485f27 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.725212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb873e-9b61-49bd-8a45-a36547485f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.725716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb873e-9b61-49bd-8a45-a36547485f27 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.726591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb873e-9b61-49bd-8a45-a36547485f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:34.728854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb873e-9b61-49bd-8a45-a36547485f27 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:34.772333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 782/1544] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:34 2022] POST /identity/v3/users => generated 302 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:34.777256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678349ef-05c5-4327-9d77-e812de335fb3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:34.780262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75448666-4277-4d48-9385-f175f987a25b None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.781080 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75448666-4277-4d48-9385-f175f987a25b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.781789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75448666-4277-4d48-9385-f175f987a25b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.799951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678349ef-05c5-4327-9d77-e812de335fb3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:34.807778 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-75448666-4277-4d48-9385-f175f987a25b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:34.809338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-75448666-4277-4d48-9385-f175f987a25b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:34.826612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 783/1545] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:42:34.832424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300652e-c2e2-4fad-872a-bc85fdd4f7a9 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.833008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300652e-c2e2-4fad-872a-bc85fdd4f7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.833454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b8518b-89e9-475e-b9ea-5224ec28ccb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:34.833835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2300652e-c2e2-4fad-872a-bc85fdd4f7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.834085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8518b-89e9-475e-b9ea-5224ec28ccb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.834366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8518b-89e9-475e-b9ea-5224ec28ccb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.834636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b8518b-89e9-475e-b9ea-5224ec28ccb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.835618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8518b-89e9-475e-b9ea-5224ec28ccb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:34.837408 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b8518b-89e9-475e-b9ea-5224ec28ccb7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:34.864216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 763/1546] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:34.864216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecec41-bf82-4228-bd3c-75a58e7732d9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.864216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecec41-bf82-4228-bd3c-75a58e7732d9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.864216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ecec41-bf82-4228-bd3c-75a58e7732d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.866230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2300652e-c2e2-4fad-872a-bc85fdd4f7a9 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:34.867676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2300652e-c2e2-4fad-872a-bc85fdd4f7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:34.877666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 784/1547] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:42:34.878976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300652e-c2e2-4fad-872a-bc85fdd4f7a9 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:34.889576 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2300652e-c2e2-4fad-872a-bc85fdd4f7a9 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:34.900228 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ffa5d-7df7-4ccf-9431-32237429a8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:34.901340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ffa5d-7df7-4ccf-9431-32237429a8d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.901850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ffa5d-7df7-4ccf-9431-32237429a8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.902343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ffa5d-7df7-4ccf-9431-32237429a8d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.903216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ffa5d-7df7-4ccf-9431-32237429a8d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:34.905048 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ffa5d-7df7-4ccf-9431-32237429a8d3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:34.916320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-90ecec41-bf82-4228-bd3c-75a58e7732d9 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:34.916320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-90ecec41-bf82-4228-bd3c-75a58e7732d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:34.923706 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 785/1548] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:34.924951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ffa5d-7df7-4ccf-9431-32237429a8d3 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:34.940134 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ffa5d-7df7-4ccf-9431-32237429a8d3 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:34.951320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 764/1549] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:34.952262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecec41-bf82-4228-bd3c-75a58e7732d9 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:34.954233 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7034ce6-a81f-4441-a9fe-d2f572ea1028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:34.955403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034ce6-a81f-4441-a9fe-d2f572ea1028 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.955907 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034ce6-a81f-4441-a9fe-d2f572ea1028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.956493 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7034ce6-a81f-4441-a9fe-d2f572ea1028 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.957393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7034ce6-a81f-4441-a9fe-d2f572ea1028 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:42:34.958154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7034ce6-a81f-4441-a9fe-d2f572ea1028 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:34.964899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7034ce6-a81f-4441-a9fe-d2f572ea1028 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:34.971539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ecec41-bf82-4228-bd3c-75a58e7732d9 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:34.980124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 786/1550] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:42:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:42:34.981988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715cf18-e33e-4760-8826-a187d6ff361e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.982498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715cf18-e33e-4760-8826-a187d6ff361e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.983002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e715cf18-e33e-4760-8826-a187d6ff361e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:34.988729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f5ea60-1744-4c45-b02b-91ee777b5a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:34.989509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5ea60-1744-4c45-b02b-91ee777b5a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:34.989826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5ea60-1744-4c45-b02b-91ee777b5a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:34.990101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f5ea60-1744-4c45-b02b-91ee777b5a6e admin admin] PATH_INFO: `/v3/projects/2ed669dd0269435a97ca38c4a869e50e/users/ab043ae6f1964e4eba35ca45ef5e53cb/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.005929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e715cf18-e33e-4760-8826-a187d6ff361e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.007533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e715cf18-e33e-4760-8826-a187d6ff361e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.018425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 787/1551] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.019213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5ea60-1744-4c45-b02b-91ee777b5a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ed669dd0269435a97ca38c4a869e50e, user_id=ab043ae6f1964e4eba35ca45ef5e53cb, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.020585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e182ca5-e58a-4366-aeeb-7f511ff382c8 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.021122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e182ca5-e58a-4366-aeeb-7f511ff382c8 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.021690 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e182ca5-e58a-4366-aeeb-7f511ff382c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.021918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f5ea60-1744-4c45-b02b-91ee777b5a6e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.043522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 765/1552] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:34 2022] PUT /identity/v3/projects/2ed669dd0269435a97ca38c4a869e50e/users/ab043ae6f1964e4eba35ca45ef5e53cb/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:35.043522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40f90e-d39d-4dcd-8331-4b054611cd88 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.043522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40f90e-d39d-4dcd-8331-4b054611cd88 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.043522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb40f90e-d39d-4dcd-8331-4b054611cd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.049902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-3e182ca5-e58a-4366-aeeb-7f511ff382c8 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.051439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3e182ca5-e58a-4366-aeeb-7f511ff382c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.066254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-bb40f90e-d39d-4dcd-8331-4b054611cd88 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.067978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bb40f90e-d39d-4dcd-8331-4b054611cd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.118219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 788/1553] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.119097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e182ca5-e58a-4366-aeeb-7f511ff382c8 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.129168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e182ca5-e58a-4366-aeeb-7f511ff382c8 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.133319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 766/1554] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.133819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9feb305-f6a0-475e-b957-14dce64fa5b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.134419 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb305-f6a0-475e-b957-14dce64fa5b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.134696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb305-f6a0-475e-b957-14dce64fa5b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.134813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876db37-6c60-4a29-811e-73c51c0aa029 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.135030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9feb305-f6a0-475e-b957-14dce64fa5b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.135144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876db37-6c60-4a29-811e-73c51c0aa029 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.135355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0876db37-6c60-4a29-811e-73c51c0aa029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.135582 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feb305-f6a0-475e-b957-14dce64fa5b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.136586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: WARNING py.warnings [None req-f9feb305-f6a0-475e-b957-14dce64fa5b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:42:35.136586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: warnings.warn(msg) Sep 06 22:42:35.136586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: Sep 06 22:42:35.137862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9feb305-f6a0-475e-b957-14dce64fa5b1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.156180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0876db37-6c60-4a29-811e-73c51c0aa029 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.158431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0876db37-6c60-4a29-811e-73c51c0aa029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.166756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 767/1555] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.167931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f9baf-40ad-4b1f-b8ac-5499ace36016 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.168214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f9baf-40ad-4b1f-b8ac-5499ace36016 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.168482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673f9baf-40ad-4b1f-b8ac-5499ace36016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.171714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 789/1556] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/projects => generated 371 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:35.173310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5a683-3412-4e3c-a294-98fd8b1f7bda None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.173594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5a683-3412-4e3c-a294-98fd8b1f7bda None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.173871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac5a683-3412-4e3c-a294-98fd8b1f7bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.181988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:35.190187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-673f9baf-40ad-4b1f-b8ac-5499ace36016 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.191385 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-673f9baf-40ad-4b1f-b8ac-5499ace36016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.194115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-cac5a683-3412-4e3c-a294-98fd8b1f7bda None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.195321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-cac5a683-3412-4e3c-a294-98fd8b1f7bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.252483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 768/1557] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.256191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f09779-ddff-479d-8fcc-0f5dcf7b729a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.256191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f09779-ddff-479d-8fcc-0f5dcf7b729a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.256191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f09779-ddff-479d-8fcc-0f5dcf7b729a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.258518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 790/1558] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.259791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d642e8f-ce20-43bb-a69e-040f6e96d71b None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.260075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d642e8f-ce20-43bb-a69e-040f6e96d71b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.260349 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d642e8f-ce20-43bb-a69e-040f6e96d71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.278399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-25f09779-ddff-479d-8fcc-0f5dcf7b729a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.279599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-25f09779-ddff-479d-8fcc-0f5dcf7b729a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.280935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-5d642e8f-ce20-43bb-a69e-040f6e96d71b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.282264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5d642e8f-ce20-43bb-a69e-040f6e96d71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.287487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 769/1559] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.288253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f09779-ddff-479d-8fcc-0f5dcf7b729a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.297953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f09779-ddff-479d-8fcc-0f5dcf7b729a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.298262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 791/1560] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.299492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce03f3-dddc-454a-9ddb-12d6707d5142 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.299935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce03f3-dddc-454a-9ddb-12d6707d5142 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.300209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdce03f3-dddc-454a-9ddb-12d6707d5142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.305737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f9a23-16d2-4ab8-b1a3-14180f59c091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.306564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f9a23-16d2-4ab8-b1a3-14180f59c091 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.306841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f9a23-16d2-4ab8-b1a3-14180f59c091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.307110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f9a23-16d2-4ab8-b1a3-14180f59c091 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.307755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f9a23-16d2-4ab8-b1a3-14180f59c091 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:42:35.308050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f9a23-16d2-4ab8-b1a3-14180f59c091 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.309130 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-215f9a23-16d2-4ab8-b1a3-14180f59c091 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:42:35.309130 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 06 22:42:35.309130 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: Sep 06 22:42:35.309998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215f9a23-16d2-4ab8-b1a3-14180f59c091 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.312540 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:35.320468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-cdce03f3-dddc-454a-9ddb-12d6707d5142 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.321704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-cdce03f3-dddc-454a-9ddb-12d6707d5142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.324737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 770/1561] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:42:35.325411 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f9a23-16d2-4ab8-b1a3-14180f59c091 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.336032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215f9a23-16d2-4ab8-b1a3-14180f59c091 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.337242 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 792/1562] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.338582 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecd613-663b-4e05-a597-ca3ef6416c61 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.338867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecd613-663b-4e05-a597-ca3ef6416c61 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.339136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecd613-663b-4e05-a597-ca3ef6416c61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.341122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86280e8-adb5-44d6-b371-a9e0f6e319bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.341722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86280e8-adb5-44d6-b371-a9e0f6e319bd admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.342003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86280e8-adb5-44d6-b371-a9e0f6e319bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.342274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86280e8-adb5-44d6-b371-a9e0f6e319bd admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.342890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86280e8-adb5-44d6-b371-a9e0f6e319bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.343897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-e86280e8-adb5-44d6-b371-a9e0f6e319bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:42:35.343897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 06 22:42:35.343897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: Sep 06 22:42:35.345632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86280e8-adb5-44d6-b371-a9e0f6e319bd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.360984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2cecd613-663b-4e05-a597-ca3ef6416c61 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.362201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2cecd613-663b-4e05-a597-ca3ef6416c61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.368056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 771/1563] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/users => generated 286 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:35.369081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86280e8-adb5-44d6-b371-a9e0f6e319bd admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.370663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 793/1564] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.371445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecd613-663b-4e05-a597-ca3ef6416c61 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.381374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86280e8-adb5-44d6-b371-a9e0f6e319bd admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.386633 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5141e5c-43da-46df-9c4b-1232bd162d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.387232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141e5c-43da-46df-9c4b-1232bd162d7f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.387513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141e5c-43da-46df-9c4b-1232bd162d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.387786 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5141e5c-43da-46df-9c4b-1232bd162d7f admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.388425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5141e5c-43da-46df-9c4b-1232bd162d7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:42:35.388725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5141e5c-43da-46df-9c4b-1232bd162d7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.390346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5141e5c-43da-46df-9c4b-1232bd162d7f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.409728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 772/1565] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:42:35.409728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581f0b2-3944-4790-993a-bd1516f0eac0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.409728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581f0b2-3944-4790-993a-bd1516f0eac0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.409728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0581f0b2-3944-4790-993a-bd1516f0eac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.427031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0581f0b2-3944-4790-993a-bd1516f0eac0 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.428233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0581f0b2-3944-4790-993a-bd1516f0eac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.429369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecd613-663b-4e05-a597-ca3ef6416c61 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.433007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c294153-839e-4955-bf5e-748eccda096f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.433615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c294153-839e-4955-bf5e-748eccda096f service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.433929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c294153-839e-4955-bf5e-748eccda096f service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.434156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c294153-839e-4955-bf5e-748eccda096f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.436340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 773/1566] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.437316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f112db-eb1a-4fa3-8316-628e9c720eaa None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.437594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f112db-eb1a-4fa3-8316-628e9c720eaa None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.438299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f112db-eb1a-4fa3-8316-628e9c720eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.447164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c294153-839e-4955-bf5e-748eccda096f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.448624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c294153-839e-4955-bf5e-748eccda096f service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.457419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-69f112db-eb1a-4fa3-8316-628e9c720eaa None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:35.458590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-69f112db-eb1a-4fa3-8316-628e9c720eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:35.458806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 794/1567] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:35.460687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c294153-839e-4955-bf5e-748eccda096f service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.466673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 774/1568] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:35.467463 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f112db-eb1a-4fa3-8316-628e9c720eaa None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.474953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c294153-839e-4955-bf5e-748eccda096f service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.477225 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f112db-eb1a-4fa3-8316-628e9c720eaa None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.479932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666ed40-9741-4d3a-a852-a25cea615716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.480539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666ed40-9741-4d3a-a852-a25cea615716 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.480852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666ed40-9741-4d3a-a852-a25cea615716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.481134 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b666ed40-9741-4d3a-a852-a25cea615716 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.481769 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666ed40-9741-4d3a-a852-a25cea615716 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.481888 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311864f5-9209-4841-91c4-56c0f1727270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.482489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311864f5-9209-4841-91c4-56c0f1727270 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.482760 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311864f5-9209-4841-91c4-56c0f1727270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.483018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311864f5-9209-4841-91c4-56c0f1727270 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.483654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311864f5-9209-4841-91c4-56c0f1727270 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:42:35.483865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b666ed40-9741-4d3a-a852-a25cea615716 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.483966 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311864f5-9209-4841-91c4-56c0f1727270 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.486413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311864f5-9209-4841-91c4-56c0f1727270 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.498976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 775/1569] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:42:35.499801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311864f5-9209-4841-91c4-56c0f1727270 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.511195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311864f5-9209-4841-91c4-56c0f1727270 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.514776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 795/1570] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:35.515460 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666ed40-9741-4d3a-a852-a25cea615716 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.519414 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62799cf-7615-4f49-9777-c191ac724f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.519864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62799cf-7615-4f49-9777-c191ac724f86 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.520144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62799cf-7615-4f49-9777-c191ac724f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.520405 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62799cf-7615-4f49-9777-c191ac724f86 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.521091 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62799cf-7615-4f49-9777-c191ac724f86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.522684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62799cf-7615-4f49-9777-c191ac724f86 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.534711 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 776/1571] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:35.535410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62799cf-7615-4f49-9777-c191ac724f86 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.545371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62799cf-7615-4f49-9777-c191ac724f86 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.600072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b666ed40-9741-4d3a-a852-a25cea615716 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.610092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107514de-e4cb-43f1-ac92-7a43adbdc5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.611156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107514de-e4cb-43f1-ac92-7a43adbdc5db admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.611913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107514de-e4cb-43f1-ac92-7a43adbdc5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.612511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107514de-e4cb-43f1-ac92-7a43adbdc5db admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.613351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107514de-e4cb-43f1-ac92-7a43adbdc5db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:42:35.613796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107514de-e4cb-43f1-ac92-7a43adbdc5db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.615449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107514de-e4cb-43f1-ac92-7a43adbdc5db admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.616367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701f98a-126a-4ab0-9887-802c8d4801f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.617087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f98a-126a-4ab0-9887-802c8d4801f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.617445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f98a-126a-4ab0-9887-802c8d4801f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.617949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701f98a-126a-4ab0-9887-802c8d4801f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.619165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701f98a-126a-4ab0-9887-802c8d4801f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.621221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6701f98a-126a-4ab0-9887-802c8d4801f7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.625023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:35.635897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 796/1572] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:42:35.635897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107514de-e4cb-43f1-ac92-7a43adbdc5db admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.649750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107514de-e4cb-43f1-ac92-7a43adbdc5db admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.654193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 777/1573] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/projects => generated 387 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:35.654720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701f98a-126a-4ab0-9887-802c8d4801f7 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.662054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dec3e4-fa99-4f54-8089-ac8ea75908f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.662054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dec3e4-fa99-4f54-8089-ac8ea75908f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.662054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dec3e4-fa99-4f54-8089-ac8ea75908f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.662054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dec3e4-fa99-4f54-8089-ac8ea75908f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.663520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dec3e4-fa99-4f54-8089-ac8ea75908f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:42:35.663520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dec3e4-fa99-4f54-8089-ac8ea75908f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.663520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dec3e4-fa99-4f54-8089-ac8ea75908f2 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.683162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 797/1574] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:42:35.684221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dec3e4-fa99-4f54-8089-ac8ea75908f2 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.701840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dec3e4-fa99-4f54-8089-ac8ea75908f2 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.736181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701f98a-126a-4ab0-9887-802c8d4801f7 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.741388 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c0b98-9bd4-4c4e-b85f-13d923e0f2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.742167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0b98-9bd4-4c4e-b85f-13d923e0f2b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.742375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0b98-9bd4-4c4e-b85f-13d923e0f2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.742706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c0b98-9bd4-4c4e-b85f-13d923e0f2b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.743452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c0b98-9bd4-4c4e-b85f-13d923e0f2b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:42:35.743818 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c0b98-9bd4-4c4e-b85f-13d923e0f2b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.745481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315c0b98-9bd4-4c4e-b85f-13d923e0f2b2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.764746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 778/1575] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:42:35.766153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c0b98-9bd4-4c4e-b85f-13d923e0f2b2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.777828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c0b98-9bd4-4c4e-b85f-13d923e0f2b2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.783441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602c67e-2cdd-44e3-9a29-02c9ad4e1906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.784072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602c67e-2cdd-44e3-9a29-02c9ad4e1906 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.784347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602c67e-2cdd-44e3-9a29-02c9ad4e1906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.784617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6602c67e-2cdd-44e3-9a29-02c9ad4e1906 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.785275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602c67e-2cdd-44e3-9a29-02c9ad4e1906 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.786787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb847aa-7533-44d4-818a-615f7da030f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.787222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6602c67e-2cdd-44e3-9a29-02c9ad4e1906 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.787536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb847aa-7533-44d4-818a-615f7da030f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.787878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb847aa-7533-44d4-818a-615f7da030f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.788216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb847aa-7533-44d4-818a-615f7da030f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.788945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb847aa-7533-44d4-818a-615f7da030f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.790884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb847aa-7533-44d4-818a-615f7da030f4 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.795607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:35.822367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 779/1576] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/users => generated 288 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:35.826525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602c67e-2cdd-44e3-9a29-02c9ad4e1906 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.827505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 798/1577] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/projects => generated 393 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:35.833514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb847aa-7533-44d4-818a-615f7da030f4 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.846943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6602c67e-2cdd-44e3-9a29-02c9ad4e1906 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.914686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb847aa-7533-44d4-818a-615f7da030f4 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.916712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8be420-6fe7-4684-859f-caf589d4c939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.917472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8be420-6fe7-4684-859f-caf589d4c939 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.917817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8be420-6fe7-4684-859f-caf589d4c939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.918275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8be420-6fe7-4684-859f-caf589d4c939 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.919053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8be420-6fe7-4684-859f-caf589d4c939 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.919207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb87095-d0b3-4960-a3de-626bddf66cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.919820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb87095-d0b3-4960-a3de-626bddf66cd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.920101 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb87095-d0b3-4960-a3de-626bddf66cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.920358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb87095-d0b3-4960-a3de-626bddf66cd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.921041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb87095-d0b3-4960-a3de-626bddf66cd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.921388 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8be420-6fe7-4684-859f-caf589d4c939 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.926610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb87095-d0b3-4960-a3de-626bddf66cd0 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.937260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 780/1578] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:35 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:35.942851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8be420-6fe7-4684-859f-caf589d4c939 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.954401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8be420-6fe7-4684-859f-caf589d4c939 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:35.958268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 799/1579] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:35.959594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ab8fd-e85d-44b7-b211-ae505a4d97de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:35.960214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ab8fd-e85d-44b7-b211-ae505a4d97de admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:35.960473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ab8fd-e85d-44b7-b211-ae505a4d97de admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:35.960742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050ab8fd-e85d-44b7-b211-ae505a4d97de admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:35.961591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ab8fd-e85d-44b7-b211-ae505a4d97de admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:35.962208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb87095-d0b3-4960-a3de-626bddf66cd0 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:35.963615 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050ab8fd-e85d-44b7-b211-ae505a4d97de admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:35.990216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 781/1580] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/users => generated 294 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:35.991468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ab8fd-e85d-44b7-b211-ae505a4d97de admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.007962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050ab8fd-e85d-44b7-b211-ae505a4d97de admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.031919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:36.036934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb87095-d0b3-4960-a3de-626bddf66cd0 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.044916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267738c-c63f-4250-9464-40d45460cdd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.045909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267738c-c63f-4250-9464-40d45460cdd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.046235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267738c-c63f-4250-9464-40d45460cdd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.046516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267738c-c63f-4250-9464-40d45460cdd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.047371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267738c-c63f-4250-9464-40d45460cdd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.049358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267738c-c63f-4250-9464-40d45460cdd6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.078081 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 800/1581] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/projects => generated 377 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:36.079705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267738c-c63f-4250-9464-40d45460cdd6 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.085678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fd1a4-b7b1-4606-8393-681e88abd082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.086496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fd1a4-b7b1-4606-8393-681e88abd082 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.087268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fd1a4-b7b1-4606-8393-681e88abd082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.087268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539fd1a4-b7b1-4606-8393-681e88abd082 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.087718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fd1a4-b7b1-4606-8393-681e88abd082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.089653 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539fd1a4-b7b1-4606-8393-681e88abd082 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.121084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 782/1582] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:35 2022] POST /identity/v3/projects => generated 375 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:36.121827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fd1a4-b7b1-4606-8393-681e88abd082 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.151981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267738c-c63f-4250-9464-40d45460cdd6 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.187290 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:36.205240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539fd1a4-b7b1-4606-8393-681e88abd082 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.211404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043aafa8-6996-4257-9061-a6dba7a475eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.212016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aafa8-6996-4257-9061-a6dba7a475eb admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.212836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aafa8-6996-4257-9061-a6dba7a475eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.212836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043aafa8-6996-4257-9061-a6dba7a475eb admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.213202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043aafa8-6996-4257-9061-a6dba7a475eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.215117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043aafa8-6996-4257-9061-a6dba7a475eb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.222925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6dd91-4140-4e1c-bd86-de311e703111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.223630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6dd91-4140-4e1c-bd86-de311e703111 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.223933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6dd91-4140-4e1c-bd86-de311e703111 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.224316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed6dd91-4140-4e1c-bd86-de311e703111 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.225897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6dd91-4140-4e1c-bd86-de311e703111 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.228546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed6dd91-4140-4e1c-bd86-de311e703111 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.240537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 801/1583] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.241394 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6dd91-4140-4e1c-bd86-de311e703111 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.244867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 783/1584] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:36 2022] POST /identity/v3/users => generated 297 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:36.244867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043aafa8-6996-4257-9061-a6dba7a475eb admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.252596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed6dd91-4140-4e1c-bd86-de311e703111 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.256326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043aafa8-6996-4257-9061-a6dba7a475eb admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.259030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28625e16-2bf0-4518-8ba5-79dbb4fbcacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.259920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28625e16-2bf0-4518-8ba5-79dbb4fbcacc admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.260201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28625e16-2bf0-4518-8ba5-79dbb4fbcacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.260471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28625e16-2bf0-4518-8ba5-79dbb4fbcacc admin admin] PATH_INFO: `/v3/projects/f13784b3824c495193bbf96fb6a2246d/users/cd053e1b142b438cb09435855a7a6b65/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.270319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4167a-88e1-420c-9170-dea08b3e6af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.271798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4167a-88e1-420c-9170-dea08b3e6af7 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.272418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4167a-88e1-420c-9170-dea08b3e6af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.273424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf4167a-88e1-420c-9170-dea08b3e6af7 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.275804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4167a-88e1-420c-9170-dea08b3e6af7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.278468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28625e16-2bf0-4518-8ba5-79dbb4fbcacc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f13784b3824c495193bbf96fb6a2246d, user_id=cd053e1b142b438cb09435855a7a6b65, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.278919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf4167a-88e1-420c-9170-dea08b3e6af7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.279519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: WARNING py.warnings [None req-28625e16-2bf0-4518-8ba5-79dbb4fbcacc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:42:36.279519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: warnings.warn(msg) Sep 06 22:42:36.279519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: Sep 06 22:42:36.281493 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28625e16-2bf0-4518-8ba5-79dbb4fbcacc admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.306714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 802/1585] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:36 2022] PUT /identity/v3/projects/f13784b3824c495193bbf96fb6a2246d/users/cd053e1b142b438cb09435855a7a6b65/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:36.317284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 784/1586] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:36 2022] POST /identity/v3/users => generated 293 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:36.317435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28625e16-2bf0-4518-8ba5-79dbb4fbcacc admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.317528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4167a-88e1-420c-9170-dea08b3e6af7 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.331116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf4167a-88e1-420c-9170-dea08b3e6af7 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.385653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28625e16-2bf0-4518-8ba5-79dbb4fbcacc admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.390459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d569781-e3e2-4a2c-9c48-3e8126b0eb73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.391571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d569781-e3e2-4a2c-9c48-3e8126b0eb73 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.391841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d569781-e3e2-4a2c-9c48-3e8126b0eb73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.392123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d569781-e3e2-4a2c-9c48-3e8126b0eb73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.392971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df43122-4f3d-4c18-ad39-fbbbf3c0aaa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.393415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d569781-e3e2-4a2c-9c48-3e8126b0eb73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.393607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df43122-4f3d-4c18-ad39-fbbbf3c0aaa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.393863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df43122-4f3d-4c18-ad39-fbbbf3c0aaa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.394132 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df43122-4f3d-4c18-ad39-fbbbf3c0aaa8 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.394768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df43122-4f3d-4c18-ad39-fbbbf3c0aaa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.394876 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d569781-e3e2-4a2c-9c48-3e8126b0eb73 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.396728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df43122-4f3d-4c18-ad39-fbbbf3c0aaa8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.402808 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:36.411159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 803/1587] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.412449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d569781-e3e2-4a2c-9c48-3e8126b0eb73 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.426181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d569781-e3e2-4a2c-9c48-3e8126b0eb73 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.426895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 785/1588] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:36 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:36.427990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df43122-4f3d-4c18-ad39-fbbbf3c0aaa8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.433915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a1e50c-852b-471f-8940-83a25b0d327b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.434736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1e50c-852b-471f-8940-83a25b0d327b admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.435195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1e50c-852b-471f-8940-83a25b0d327b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.435657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a1e50c-852b-471f-8940-83a25b0d327b admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.436277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1e50c-852b-471f-8940-83a25b0d327b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.438404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a1e50c-852b-471f-8940-83a25b0d327b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.439492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df43122-4f3d-4c18-ad39-fbbbf3c0aaa8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.444482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd7d24-36fd-4262-b293-644c3d3bb786 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.445222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd7d24-36fd-4262-b293-644c3d3bb786 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.445566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd7d24-36fd-4262-b293-644c3d3bb786 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.445871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbd7d24-36fd-4262-b293-644c3d3bb786 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.446629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd7d24-36fd-4262-b293-644c3d3bb786 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.448218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbd7d24-36fd-4262-b293-644c3d3bb786 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.460428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 786/1589] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.461421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd7d24-36fd-4262-b293-644c3d3bb786 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.469409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 804/1590] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:36 2022] POST /identity/v3/users => generated 288 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:36.471087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f22201-c740-421f-8f3f-7cb081182189 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.471465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f22201-c740-421f-8f3f-7cb081182189 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.471846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f22201-c740-421f-8f3f-7cb081182189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.476376 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbd7d24-36fd-4262-b293-644c3d3bb786 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.482240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298b377-6a85-474e-98b1-aa2e529f2a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.482837 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298b377-6a85-474e-98b1-aa2e529f2a47 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.483116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298b377-6a85-474e-98b1-aa2e529f2a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.483388 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298b377-6a85-474e-98b1-aa2e529f2a47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.484024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298b377-6a85-474e-98b1-aa2e529f2a47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.485983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2298b377-6a85-474e-98b1-aa2e529f2a47 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.496283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 787/1591] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.497507 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf4931-0eb1-4652-a3df-5a1d5a73ab4e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.497785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf4931-0eb1-4652-a3df-5a1d5a73ab4e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.498053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abf4931-0eb1-4652-a3df-5a1d5a73ab4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.503037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c1f22201-c740-421f-8f3f-7cb081182189 None None] MFA Rules not processed for user `ab043ae6f1964e4eba35ca45ef5e53cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:36.504389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c1f22201-c740-421f-8f3f-7cb081182189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:36.527117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8abf4931-0eb1-4652-a3df-5a1d5a73ab4e None None] MFA Rules not processed for user `cd053e1b142b438cb09435855a7a6b65`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:36.528424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8abf4931-0eb1-4652-a3df-5a1d5a73ab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:36.569866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 805/1592] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:36 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:36.570513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f22201-c740-421f-8f3f-7cb081182189 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.587405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f22201-c740-421f-8f3f-7cb081182189 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.591916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d00ac-e1fa-404d-8942-ffc9f6890df1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.592519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d00ac-e1fa-404d-8942-ffc9f6890df1 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.592824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d00ac-e1fa-404d-8942-ffc9f6890df1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.593097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73d00ac-e1fa-404d-8942-ffc9f6890df1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.593731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d00ac-e1fa-404d-8942-ffc9f6890df1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.595990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73d00ac-e1fa-404d-8942-ffc9f6890df1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.605498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 788/1593] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:36 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:36.606061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 806/1594] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.606170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf4931-0eb1-4652-a3df-5a1d5a73ab4e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.606752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d00ac-e1fa-404d-8942-ffc9f6890df1 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.616539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73d00ac-e1fa-404d-8942-ffc9f6890df1 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.616989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abf4931-0eb1-4652-a3df-5a1d5a73ab4e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.621188 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fd3f2-0e5a-4dcd-85af-1e53cb51d8e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.621609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2dfcf2-1ca0-4f08-aa20-6b6e9834e663 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.621766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fd3f2-0e5a-4dcd-85af-1e53cb51d8e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.622076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fd3f2-0e5a-4dcd-85af-1e53cb51d8e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.622236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dfcf2-1ca0-4f08-aa20-6b6e9834e663 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.622333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502fd3f2-0e5a-4dcd-85af-1e53cb51d8e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.622503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dfcf2-1ca0-4f08-aa20-6b6e9834e663 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.622775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2dfcf2-1ca0-4f08-aa20-6b6e9834e663 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.622956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fd3f2-0e5a-4dcd-85af-1e53cb51d8e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.623413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2dfcf2-1ca0-4f08-aa20-6b6e9834e663 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.624526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502fd3f2-0e5a-4dcd-85af-1e53cb51d8e9 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.625024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2dfcf2-1ca0-4f08-aa20-6b6e9834e663 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.634980 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 807/1595] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.638599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 789/1596] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.638755 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fd3f2-0e5a-4dcd-85af-1e53cb51d8e9 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.638871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2dfcf2-1ca0-4f08-aa20-6b6e9834e663 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.645977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2dfcf2-1ca0-4f08-aa20-6b6e9834e663 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.648689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502fd3f2-0e5a-4dcd-85af-1e53cb51d8e9 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.650866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706c542-0087-49a1-a70e-8957e96af826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.651449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706c542-0087-49a1-a70e-8957e96af826 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.651720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706c542-0087-49a1-a70e-8957e96af826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.651984 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8706c542-0087-49a1-a70e-8957e96af826 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.652601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706c542-0087-49a1-a70e-8957e96af826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.653443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4faa2-81da-49df-9862-926cebb4150a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.654090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4faa2-81da-49df-9862-926cebb4150a admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.654260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8706c542-0087-49a1-a70e-8957e96af826 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.654377 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4faa2-81da-49df-9862-926cebb4150a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.654637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4faa2-81da-49df-9862-926cebb4150a admin admin] PATH_INFO: `/v3/projects/817010ab38864d8aafb61140eb176da1/users/4dd9f0bff3044d7db561dff21ac0e7ff/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.659911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:36.669063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 790/1597] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.669906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706c542-0087-49a1-a70e-8957e96af826 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.671366 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4faa2-81da-49df-9862-926cebb4150a admin admin] RBAC: Authorizing `identity:create_grant(project_id=817010ab38864d8aafb61140eb176da1, user_id=4dd9f0bff3044d7db561dff21ac0e7ff, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.674172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec4faa2-81da-49df-9862-926cebb4150a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.680674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8706c542-0087-49a1-a70e-8957e96af826 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.687182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 808/1598] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:36 2022] PUT /identity/v3/projects/817010ab38864d8aafb61140eb176da1/users/4dd9f0bff3044d7db561dff21ac0e7ff/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:36.688514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4faa2-81da-49df-9862-926cebb4150a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.741172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d055a36-c5a2-45d4-be91-ebb00a42c8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.741784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d055a36-c5a2-45d4-be91-ebb00a42c8f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.742060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d055a36-c5a2-45d4-be91-ebb00a42c8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.742331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d055a36-c5a2-45d4-be91-ebb00a42c8f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.742965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d055a36-c5a2-45d4-be91-ebb00a42c8f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.744544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d055a36-c5a2-45d4-be91-ebb00a42c8f4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.749771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec4faa2-81da-49df-9862-926cebb4150a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.754717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 791/1599] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.755085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773040ba-1e3b-4269-b1cd-e261a042ceec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.755504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d055a36-c5a2-45d4-be91-ebb00a42c8f4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.755791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773040ba-1e3b-4269-b1cd-e261a042ceec admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.756066 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773040ba-1e3b-4269-b1cd-e261a042ceec admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.756340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773040ba-1e3b-4269-b1cd-e261a042ceec admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.756980 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773040ba-1e3b-4269-b1cd-e261a042ceec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.759239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773040ba-1e3b-4269-b1cd-e261a042ceec admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.765051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d055a36-c5a2-45d4-be91-ebb00a42c8f4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.770913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9be4e9-00ca-48dc-b54a-cb15fba0acf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.771498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9be4e9-00ca-48dc-b54a-cb15fba0acf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.771776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9be4e9-00ca-48dc-b54a-cb15fba0acf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.772047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9be4e9-00ca-48dc-b54a-cb15fba0acf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.772677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9be4e9-00ca-48dc-b54a-cb15fba0acf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.774259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9be4e9-00ca-48dc-b54a-cb15fba0acf3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.784775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 792/1600] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.785399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9be4e9-00ca-48dc-b54a-cb15fba0acf3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.790104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 809/1601] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:36 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:36.790971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773040ba-1e3b-4269-b1cd-e261a042ceec admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.795307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9be4e9-00ca-48dc-b54a-cb15fba0acf3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.800457 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:36.851437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d023b0-3be3-424f-964a-9ed82a88975f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.852042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d023b0-3be3-424f-964a-9ed82a88975f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.852318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d023b0-3be3-424f-964a-9ed82a88975f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.852591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d023b0-3be3-424f-964a-9ed82a88975f admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.853242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d023b0-3be3-424f-964a-9ed82a88975f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.855128 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d023b0-3be3-424f-964a-9ed82a88975f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.855771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773040ba-1e3b-4269-b1cd-e261a042ceec admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.860628 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3beab-289b-450e-9948-ef1c668a87d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.861355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3beab-289b-450e-9948-ef1c668a87d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.861632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3beab-289b-450e-9948-ef1c668a87d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.861905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce3beab-289b-450e-9948-ef1c668a87d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.862542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3beab-289b-450e-9948-ef1c668a87d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.864160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce3beab-289b-450e-9948-ef1c668a87d7 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.874896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 810/1602] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.876018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3beab-289b-450e-9948-ef1c668a87d7 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.883158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 793/1603] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:36 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:36.884062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d023b0-3be3-424f-964a-9ed82a88975f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.885799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce3beab-289b-450e-9948-ef1c668a87d7 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.944093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeea34-476b-4b75-9509-07ca09cf6231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.944692 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeea34-476b-4b75-9509-07ca09cf6231 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.945003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeea34-476b-4b75-9509-07ca09cf6231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.945272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3aeea34-476b-4b75-9509-07ca09cf6231 admin admin] PATH_INFO: `/v3/projects/38bf7d2d331a472c9d517f3c3067f58e/users/e305e7d0cbfc4aeca6164685d0155475/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.947978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d023b0-3be3-424f-964a-9ed82a88975f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:36.952535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e76411-7ede-4c8f-9a7a-735dfaa89090 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:36.953159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76411-7ede-4c8f-9a7a-735dfaa89090 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.954064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76411-7ede-4c8f-9a7a-735dfaa89090 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.954371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e76411-7ede-4c8f-9a7a-735dfaa89090 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.955083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e76411-7ede-4c8f-9a7a-735dfaa89090 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.956668 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e76411-7ede-4c8f-9a7a-735dfaa89090 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.961614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aeea34-476b-4b75-9509-07ca09cf6231 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38bf7d2d331a472c9d517f3c3067f58e, user_id=e305e7d0cbfc4aeca6164685d0155475, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:36.964239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3aeea34-476b-4b75-9509-07ca09cf6231 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:36.967073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 794/1604] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:36 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:36.968080 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd977fd-6252-4fb2-a7ef-a4597ce3750e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:36.968356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd977fd-6252-4fb2-a7ef-a4597ce3750e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:36.968625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd977fd-6252-4fb2-a7ef-a4597ce3750e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:36.979858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 811/1605] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:36 2022] PUT /identity/v3/projects/38bf7d2d331a472c9d517f3c3067f58e/users/e305e7d0cbfc4aeca6164685d0155475/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:36.980709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeea34-476b-4b75-9509-07ca09cf6231 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:36.985596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:36.992555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0cd977fd-6252-4fb2-a7ef-a4597ce3750e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:36.993935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0cd977fd-6252-4fb2-a7ef-a4597ce3750e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:37.045103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3aeea34-476b-4b75-9509-07ca09cf6231 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.050255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bad1d-a2f6-40c9-9091-148f535af2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.050930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bad1d-a2f6-40c9-9091-148f535af2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.051191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bad1d-a2f6-40c9-9091-148f535af2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.051461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bad1d-a2f6-40c9-9091-148f535af2a7 admin admin] PATH_INFO: `/v3/projects/84c39ea86193481fa08158197629ea85/users/2945d87f1c114dd0b42c4e58a8133e76/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.054889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 795/1606] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:37.055694 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd977fd-6252-4fb2-a7ef-a4597ce3750e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.065008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd977fd-6252-4fb2-a7ef-a4597ce3750e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.069660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bfb4b-8bcb-4052-8160-e7983c00c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.070197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bad1d-a2f6-40c9-9091-148f535af2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84c39ea86193481fa08158197629ea85, user_id=2945d87f1c114dd0b42c4e58a8133e76, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.070323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfb4b-8bcb-4052-8160-e7983c00c71e admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.070589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfb4b-8bcb-4052-8160-e7983c00c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.070875 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7bfb4b-8bcb-4052-8160-e7983c00c71e admin admin] PATH_INFO: `/v3/projects/200607d5d1f8491e9168dddd9903b779/users/a742445cb56f4e1b951e76acfc16a695/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.073073 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bad1d-a2f6-40c9-9091-148f535af2a7 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.087402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bfb4b-8bcb-4052-8160-e7983c00c71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=200607d5d1f8491e9168dddd9903b779, user_id=a742445cb56f4e1b951e76acfc16a695, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.087964 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 812/1607] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:36 2022] PUT /identity/v3/projects/84c39ea86193481fa08158197629ea85/users/2945d87f1c114dd0b42c4e58a8133e76/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:37.088753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bad1d-a2f6-40c9-9091-148f535af2a7 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.090092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7bfb4b-8bcb-4052-8160-e7983c00c71e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.101925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:37.103771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 796/1608] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:37 2022] PUT /identity/v3/projects/200607d5d1f8491e9168dddd9903b779/users/a742445cb56f4e1b951e76acfc16a695/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:37.104542 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bfb4b-8bcb-4052-8160-e7983c00c71e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.155662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bad1d-a2f6-40c9-9091-148f535af2a7 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.169373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7bfb4b-8bcb-4052-8160-e7983c00c71e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.174489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77fd9c-a2b2-4791-b18a-925a3e367245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.175135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77fd9c-a2b2-4791-b18a-925a3e367245 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.175407 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77fd9c-a2b2-4791-b18a-925a3e367245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.175673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f77fd9c-a2b2-4791-b18a-925a3e367245 admin admin] PATH_INFO: `/v3/projects/2f01bb55623a4bf580008b75a9715796/users/da5264060ddd48afa8305c38a9bfa4c3/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.189631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77fd9c-a2b2-4791-b18a-925a3e367245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f01bb55623a4bf580008b75a9715796, user_id=da5264060ddd48afa8305c38a9bfa4c3, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.192263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f77fd9c-a2b2-4791-b18a-925a3e367245 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.205246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 797/1609] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:37 2022] PUT /identity/v3/projects/2f01bb55623a4bf580008b75a9715796/users/da5264060ddd48afa8305c38a9bfa4c3/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:37.209467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77fd9c-a2b2-4791-b18a-925a3e367245 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.214390 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea16348-cf2d-4053-985e-53fbdc2cb21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.215092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea16348-cf2d-4053-985e-53fbdc2cb21c admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.215411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea16348-cf2d-4053-985e-53fbdc2cb21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.215719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea16348-cf2d-4053-985e-53fbdc2cb21c admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.216402 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea16348-cf2d-4053-985e-53fbdc2cb21c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.218360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea16348-cf2d-4053-985e-53fbdc2cb21c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.240461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 813/1610] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:37 2022] POST /identity/v3/users => generated 309 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:37.241364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea16348-cf2d-4053-985e-53fbdc2cb21c admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.251870 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea16348-cf2d-4053-985e-53fbdc2cb21c admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.276841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f77fd9c-a2b2-4791-b18a-925a3e367245 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.282042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a49272-1483-4771-92d3-8198e1850fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.282641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a49272-1483-4771-92d3-8198e1850fb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.282915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a49272-1483-4771-92d3-8198e1850fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.283260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a49272-1483-4771-92d3-8198e1850fb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.283802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a49272-1483-4771-92d3-8198e1850fb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.285730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a49272-1483-4771-92d3-8198e1850fb4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.296727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:37.307638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 798/1611] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:37 2022] POST /identity/v3/users => generated 293 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:37.309162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add1c92-a38f-449e-9ef7-da042103d191 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.309607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add1c92-a38f-449e-9ef7-da042103d191 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.309945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9add1c92-a38f-449e-9ef7-da042103d191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.311506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee1815-fb0b-4980-8c72-dc329bcdbddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.312179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee1815-fb0b-4980-8c72-dc329bcdbddf admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.312448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee1815-fb0b-4980-8c72-dc329bcdbddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.313295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee1815-fb0b-4980-8c72-dc329bcdbddf admin admin] PATH_INFO: `/v3/projects/19cc2b5d06dc46d38905348059b73e9e/users/bd409aef013448cc86db813ad9769fa7/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.328015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee1815-fb0b-4980-8c72-dc329bcdbddf admin admin] RBAC: Authorizing `identity:create_grant(project_id=19cc2b5d06dc46d38905348059b73e9e, user_id=bd409aef013448cc86db813ad9769fa7, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.330797 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-9add1c92-a38f-449e-9ef7-da042103d191 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:37.331450 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee1815-fb0b-4980-8c72-dc329bcdbddf admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.332007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9add1c92-a38f-449e-9ef7-da042103d191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:37.343173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 799/1612] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:37.343173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add1c92-a38f-449e-9ef7-da042103d191 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.343173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9add1c92-a38f-449e-9ef7-da042103d191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:37.345209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 814/1613] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:37 2022] PUT /identity/v3/projects/19cc2b5d06dc46d38905348059b73e9e/users/bd409aef013448cc86db813ad9769fa7/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:37.346431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231ee1a-7155-4455-b992-f39a5ce9f891 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.346719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231ee1a-7155-4455-b992-f39a5ce9f891 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.346978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8231ee1a-7155-4455-b992-f39a5ce9f891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.366582 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-8231ee1a-7155-4455-b992-f39a5ce9f891 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:37.367781 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8231ee1a-7155-4455-b992-f39a5ce9f891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:37.393839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:37.404034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9add1c92-a38f-449e-9ef7-da042103d191 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.428813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 815/1614] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:37.430232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8882160-f27d-4c56-ab2d-42dc6cba35fd None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.430597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8882160-f27d-4c56-ab2d-42dc6cba35fd None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.430960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8882160-f27d-4c56-ab2d-42dc6cba35fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.454581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e829f87d-205e-4036-9a9d-bd91ef223db0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.455190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829f87d-205e-4036-9a9d-bd91ef223db0 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.455536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829f87d-205e-4036-9a9d-bd91ef223db0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.455731 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e829f87d-205e-4036-9a9d-bd91ef223db0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.459381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d8882160-f27d-4c56-ab2d-42dc6cba35fd None None] MFA Rules not processed for user `a742445cb56f4e1b951e76acfc16a695`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:37.460685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d8882160-f27d-4c56-ab2d-42dc6cba35fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:37.464729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829f87d-205e-4036-9a9d-bd91ef223db0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.466175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e829f87d-205e-4036-9a9d-bd91ef223db0 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.475986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 800/1615] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:37.477197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c3964-da6d-4b0e-a779-f1a447840e10 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.477514 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c3964-da6d-4b0e-a779-f1a447840e10 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.477724 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c3964-da6d-4b0e-a779-f1a447840e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.500714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-261c3964-da6d-4b0e-a779-f1a447840e10 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:37.502469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-261c3964-da6d-4b0e-a779-f1a447840e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:37.516688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 801/1616] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:37.516688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c3964-da6d-4b0e-a779-f1a447840e10 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.528511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c3964-da6d-4b0e-a779-f1a447840e10 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.534218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756c3275-7420-4f0d-ab01-18cb940bcf3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.534862 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c3275-7420-4f0d-ab01-18cb940bcf3c admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.535141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c3275-7420-4f0d-ab01-18cb940bcf3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.535416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756c3275-7420-4f0d-ab01-18cb940bcf3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.536057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c3275-7420-4f0d-ab01-18cb940bcf3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.537811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756c3275-7420-4f0d-ab01-18cb940bcf3c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.539492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 816/1617] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:37.544628 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8882160-f27d-4c56-ab2d-42dc6cba35fd None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.551307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 802/1618] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:37.553307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808541bd-7d43-46b8-8517-ec11f134bc88 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.553829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808541bd-7d43-46b8-8517-ec11f134bc88 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.554381 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808541bd-7d43-46b8-8517-ec11f134bc88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.560898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8882160-f27d-4c56-ab2d-42dc6cba35fd None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.566215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d19d38-b45b-4154-8a55-4f4c47161398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.566828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d19d38-b45b-4154-8a55-4f4c47161398 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.567101 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d19d38-b45b-4154-8a55-4f4c47161398 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.567370 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d19d38-b45b-4154-8a55-4f4c47161398 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.568004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d19d38-b45b-4154-8a55-4f4c47161398 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.569795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d19d38-b45b-4154-8a55-4f4c47161398 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.589776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 817/1619] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:37.589776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d19d38-b45b-4154-8a55-4f4c47161398 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.600663 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-808541bd-7d43-46b8-8517-ec11f134bc88 None None] MFA Rules not processed for user `bd409aef013448cc86db813ad9769fa7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:37.602014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-808541bd-7d43-46b8-8517-ec11f134bc88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:37.606221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d19d38-b45b-4154-8a55-4f4c47161398 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.612507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90972b68-9deb-4e01-a542-5d30f2232e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.613214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90972b68-9deb-4e01-a542-5d30f2232e71 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.613604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90972b68-9deb-4e01-a542-5d30f2232e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.613787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90972b68-9deb-4e01-a542-5d30f2232e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.624676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90972b68-9deb-4e01-a542-5d30f2232e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.626770 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90972b68-9deb-4e01-a542-5d30f2232e71 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.637038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 818/1620] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:37.637823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90972b68-9deb-4e01-a542-5d30f2232e71 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.647327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90972b68-9deb-4e01-a542-5d30f2232e71 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.651677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05760517-1501-4908-95d9-3080c171ee0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.652397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05760517-1501-4908-95d9-3080c171ee0c service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.653490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05760517-1501-4908-95d9-3080c171ee0c service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.653766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05760517-1501-4908-95d9-3080c171ee0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.666678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05760517-1501-4908-95d9-3080c171ee0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.668306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05760517-1501-4908-95d9-3080c171ee0c service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.684778 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 803/1621] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:37.685893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808541bd-7d43-46b8-8517-ec11f134bc88 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.696895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 819/1622] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:37.699495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808541bd-7d43-46b8-8517-ec11f134bc88 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.700904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05760517-1501-4908-95d9-3080c171ee0c service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.706031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012d2e5-d37c-4ef5-9189-1370aace1663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.706829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012d2e5-d37c-4ef5-9189-1370aace1663 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.707230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012d2e5-d37c-4ef5-9189-1370aace1663 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.707706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8012d2e5-d37c-4ef5-9189-1370aace1663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.715706 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05760517-1501-4908-95d9-3080c171ee0c service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.719444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012d2e5-d37c-4ef5-9189-1370aace1663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.721249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8012d2e5-d37c-4ef5-9189-1370aace1663 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.722795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8df12a-b898-41cd-b61e-386c6ca042af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.723569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8df12a-b898-41cd-b61e-386c6ca042af admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.723890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8df12a-b898-41cd-b61e-386c6ca042af admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.724338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8df12a-b898-41cd-b61e-386c6ca042af admin admin] PATH_INFO: `/v3/projects/492dbc3ce2a64ad48f7596bad133ab25/users/0daf3ccb6f554148b22d160e2bb77ed8/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.734213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 804/1623] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:37.735827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012d2e5-d37c-4ef5-9189-1370aace1663 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.745560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8df12a-b898-41cd-b61e-386c6ca042af admin admin] RBAC: Authorizing `identity:create_grant(project_id=492dbc3ce2a64ad48f7596bad133ab25, user_id=0daf3ccb6f554148b22d160e2bb77ed8, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.746931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8012d2e5-d37c-4ef5-9189-1370aace1663 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.748982 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8df12a-b898-41cd-b61e-386c6ca042af admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.753176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ab972-a7f4-4125-8566-7ebaf1f7f4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.754125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ab972-a7f4-4125-8566-7ebaf1f7f4a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.754495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ab972-a7f4-4125-8566-7ebaf1f7f4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.755038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ab972-a7f4-4125-8566-7ebaf1f7f4a6 admin admin] PATH_INFO: `/v3/projects/7d1a428e155e43119408c757abfe1019/users/56b54f6d03e948599d2c6406e333b676/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.757718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:37.771239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 820/1624] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:37 2022] PUT /identity/v3/projects/492dbc3ce2a64ad48f7596bad133ab25/users/0daf3ccb6f554148b22d160e2bb77ed8/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:37.772840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d979c-aca8-4f1e-b818-a919c80b411c None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.773249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d979c-aca8-4f1e-b818-a919c80b411c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.773617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287d979c-aca8-4f1e-b818-a919c80b411c None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.776184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 821/1625] 10.222.0.39 () {54 vars in 962 bytes} [Tue Sep 6 22:42:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:42:37.777499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d979c-aca8-4f1e-b818-a919c80b411c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.780562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ab972-a7f4-4125-8566-7ebaf1f7f4a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1a428e155e43119408c757abfe1019, user_id=56b54f6d03e948599d2c6406e333b676, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.784091 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ab972-a7f4-4125-8566-7ebaf1f7f4a6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.808869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 805/1626] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:37 2022] PUT /identity/v3/projects/7d1a428e155e43119408c757abfe1019/users/56b54f6d03e948599d2c6406e333b676/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:37.808869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ab972-a7f4-4125-8566-7ebaf1f7f4a6 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:37.867499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287d979c-aca8-4f1e-b818-a919c80b411c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.872140 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:37.902900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ab972-a7f4-4125-8566-7ebaf1f7f4a6 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:37.911031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a79c28-7f56-44b5-867c-bf1e650a89dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.912451 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a79c28-7f56-44b5-867c-bf1e650a89dc admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.913939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a79c28-7f56-44b5-867c-bf1e650a89dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.914531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a79c28-7f56-44b5-867c-bf1e650a89dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.915338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a79c28-7f56-44b5-867c-bf1e650a89dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.918426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a79c28-7f56-44b5-867c-bf1e650a89dc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:37.937270 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 806/1627] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:37 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:37.940241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830702f6-afa8-47e1-b4fb-048109c230e2 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.940529 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830702f6-afa8-47e1-b4fb-048109c230e2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.941134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830702f6-afa8-47e1-b4fb-048109c230e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.960693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39485288-0d8c-4298-8088-51acbf467034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:37.962146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39485288-0d8c-4298-8088-51acbf467034 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:37.962589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39485288-0d8c-4298-8088-51acbf467034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:37.963291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39485288-0d8c-4298-8088-51acbf467034 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:37.964220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39485288-0d8c-4298-8088-51acbf467034 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:37.966537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39485288-0d8c-4298-8088-51acbf467034 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:38.003541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-830702f6-afa8-47e1-b4fb-048109c230e2 None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:38.005010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-830702f6-afa8-47e1-b4fb-048109c230e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:38.016164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 822/1628] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:37 2022] POST /identity/v3/projects => generated 377 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:38.017563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39485288-0d8c-4298-8088-51acbf467034 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:38.111909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39485288-0d8c-4298-8088-51acbf467034 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:38.116397 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 807/1629] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Sep 6 22:42:37 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:38.116981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ff188-8f93-4447-9c37-765006d3dc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:38.117286 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830702f6-afa8-47e1-b4fb-048109c230e2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:38.117542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ff188-8f93-4447-9c37-765006d3dc23 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.118003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ff188-8f93-4447-9c37-765006d3dc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.118397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ff188-8f93-4447-9c37-765006d3dc23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.119216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ff188-8f93-4447-9c37-765006d3dc23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:38.121062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6ff188-8f93-4447-9c37-765006d3dc23 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:38.131196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830702f6-afa8-47e1-b4fb-048109c230e2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:38.140277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 823/1630] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:38.140473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec9d86-75bb-41bf-89b1-e45d3524c8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:38.140473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec9d86-75bb-41bf-89b1-e45d3524c8ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.140473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec9d86-75bb-41bf-89b1-e45d3524c8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.140714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ff188-8f93-4447-9c37-765006d3dc23 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:38.140859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ec9d86-75bb-41bf-89b1-e45d3524c8ed admin admin] PATH_INFO: `/v3/projects/492dbc3ce2a64ad48f7596bad133ab25/users/0daf3ccb6f554148b22d160e2bb77ed8/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.143716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec9d86-75bb-41bf-89b1-e45d3524c8ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=492dbc3ce2a64ad48f7596bad133ab25, user_id=0daf3ccb6f554148b22d160e2bb77ed8, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:38.147172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ec9d86-75bb-41bf-89b1-e45d3524c8ed admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:38.153385 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6ff188-8f93-4447-9c37-765006d3dc23 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:38.159571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 808/1631] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:38 2022] PUT /identity/v3/projects/492dbc3ce2a64ad48f7596bad133ab25/users/0daf3ccb6f554148b22d160e2bb77ed8/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:38.160141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec9d86-75bb-41bf-89b1-e45d3524c8ed admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:38.225680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:38.262263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ec9d86-75bb-41bf-89b1-e45d3524c8ed admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:38.263209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522abd2-d4e9-4e87-aef5-6506ee3108f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:38.264012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522abd2-d4e9-4e87-aef5-6506ee3108f8 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.264443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522abd2-d4e9-4e87-aef5-6506ee3108f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.264837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9522abd2-d4e9-4e87-aef5-6506ee3108f8 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.266153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522abd2-d4e9-4e87-aef5-6506ee3108f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:38.270033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9522abd2-d4e9-4e87-aef5-6506ee3108f8 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:38.270382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a588326e-1c8b-46f8-bd5f-0a4c29ebbcaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:38.271519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588326e-1c8b-46f8-bd5f-0a4c29ebbcaf service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.271803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588326e-1c8b-46f8-bd5f-0a4c29ebbcaf service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.272075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a588326e-1c8b-46f8-bd5f-0a4c29ebbcaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.303234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 824/1632] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:38 2022] POST /identity/v3/users => generated 295 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:38.304483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522abd2-d4e9-4e87-aef5-6506ee3108f8 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:38.320734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9522abd2-d4e9-4e87-aef5-6506ee3108f8 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:38.327567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013be89-add9-457c-a2c1-fbbd98d27f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:38.329133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013be89-add9-457c-a2c1-fbbd98d27f99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.329522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013be89-add9-457c-a2c1-fbbd98d27f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.329831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f013be89-add9-457c-a2c1-fbbd98d27f99 admin admin] PATH_INFO: `/v3/projects/7d1a428e155e43119408c757abfe1019/users/56b54f6d03e948599d2c6406e333b676/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.332066 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f013be89-add9-457c-a2c1-fbbd98d27f99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d1a428e155e43119408c757abfe1019, user_id=56b54f6d03e948599d2c6406e333b676, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:38.335539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f013be89-add9-457c-a2c1-fbbd98d27f99 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:38.353060 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 825/1633] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:38 2022] PUT /identity/v3/projects/7d1a428e155e43119408c757abfe1019/users/56b54f6d03e948599d2c6406e333b676/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:38.354048 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013be89-add9-457c-a2c1-fbbd98d27f99 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:38.363022 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588326e-1c8b-46f8-bd5f-0a4c29ebbcaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:38.365229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a588326e-1c8b-46f8-bd5f-0a4c29ebbcaf service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:38.372051 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:38.382858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 809/1634] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:38.384267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963632c-f7e7-4534-9540-3e3001906c0b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.384552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963632c-f7e7-4534-9540-3e3001906c0b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.384894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4963632c-f7e7-4534-9540-3e3001906c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.435823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4963632c-f7e7-4534-9540-3e3001906c0b None None] MFA Rules not processed for user `56b54f6d03e948599d2c6406e333b676`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:38.437070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4963632c-f7e7-4534-9540-3e3001906c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:38.515932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f013be89-add9-457c-a2c1-fbbd98d27f99 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:38.525656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 810/1635] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:38 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:38.528076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76880ae-33c3-452d-bff6-ec01dab19552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:38.528465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76880ae-33c3-452d-bff6-ec01dab19552 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.528738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76880ae-33c3-452d-bff6-ec01dab19552 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.529026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76880ae-33c3-452d-bff6-ec01dab19552 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.529667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76880ae-33c3-452d-bff6-ec01dab19552 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:38.531307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76880ae-33c3-452d-bff6-ec01dab19552 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:38.539559 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963632c-f7e7-4534-9540-3e3001906c0b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:38.560049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 826/1636] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:38.561156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76880ae-33c3-452d-bff6-ec01dab19552 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:38.577807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76880ae-33c3-452d-bff6-ec01dab19552 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:38.584824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922ac71-799b-484b-8969-ffea82e55f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:38.585967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922ac71-799b-484b-8969-ffea82e55f62 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.586561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922ac71-799b-484b-8969-ffea82e55f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.586999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4922ac71-799b-484b-8969-ffea82e55f62 admin admin] PATH_INFO: `/v3/projects/63e51fefc9ed4174826bff53fe6da705/users/38317a8501cc414cadb852610a8f69f2/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.606374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922ac71-799b-484b-8969-ffea82e55f62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e51fefc9ed4174826bff53fe6da705, user_id=38317a8501cc414cadb852610a8f69f2, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:38.611184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4922ac71-799b-484b-8969-ffea82e55f62 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:38.627563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4963632c-f7e7-4534-9540-3e3001906c0b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:38.635029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 827/1637] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:38 2022] PUT /identity/v3/projects/63e51fefc9ed4174826bff53fe6da705/users/38317a8501cc414cadb852610a8f69f2/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:42:38.639473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922ac71-799b-484b-8969-ffea82e55f62 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:38.655323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:38.695483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2ce94-4aab-4a8c-b099-ab35ecefa02a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:38.696242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2ce94-4aab-4a8c-b099-ab35ecefa02a service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.696621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2ce94-4aab-4a8c-b099-ab35ecefa02a service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.697024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2ce94-4aab-4a8c-b099-ab35ecefa02a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.783272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4922ac71-799b-484b-8969-ffea82e55f62 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:38.792433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2ce94-4aab-4a8c-b099-ab35ecefa02a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:38.794216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2ce94-4aab-4a8c-b099-ab35ecefa02a service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:38.794942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465bed5-9dbf-4aa0-9f25-68ae9e421706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:38.795839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465bed5-9dbf-4aa0-9f25-68ae9e421706 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.795839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465bed5-9dbf-4aa0-9f25-68ae9e421706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.796144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465bed5-9dbf-4aa0-9f25-68ae9e421706 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.796902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465bed5-9dbf-4aa0-9f25-68ae9e421706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:38.799927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465bed5-9dbf-4aa0-9f25-68ae9e421706 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:38.811788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 811/1638] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:38 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 273 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:42:38.814012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 828/1639] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:38 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:38.818712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2ce94-4aab-4a8c-b099-ab35ecefa02a service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:38.822200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0d7d3-71e1-4514-b57d-4955272f05c2 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.823285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0d7d3-71e1-4514-b57d-4955272f05c2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.823562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c0d7d3-71e1-4514-b57d-4955272f05c2 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.826390 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 829/1640] 10.222.0.39 () {54 vars in 912 bytes} [Tue Sep 6 22:42:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:42:38.834700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200ca0a-93a5-4beb-9113-c4c711ee2ab6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.835282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200ca0a-93a5-4beb-9113-c4c711ee2ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.835723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4200ca0a-93a5-4beb-9113-c4c711ee2ab6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.838055 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2ce94-4aab-4a8c-b099-ab35ecefa02a service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:38.843904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e8e6f5-41c4-4c73-94ba-6f1c08f4b30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:38.845520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8e6f5-41c4-4c73-94ba-6f1c08f4b30f admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.845848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8e6f5-41c4-4c73-94ba-6f1c08f4b30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.846193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e8e6f5-41c4-4c73-94ba-6f1c08f4b30f admin admin] PATH_INFO: `/v3/projects/63e51fefc9ed4174826bff53fe6da705/users/38317a8501cc414cadb852610a8f69f2/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.848941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8e6f5-41c4-4c73-94ba-6f1c08f4b30f admin admin] RBAC: Authorizing `identity:create_grant(project_id=63e51fefc9ed4174826bff53fe6da705, user_id=38317a8501cc414cadb852610a8f69f2, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:38.856960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e8e6f5-41c4-4c73-94ba-6f1c08f4b30f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:38.868750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-4200ca0a-93a5-4beb-9113-c4c711ee2ab6 None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:38.878889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4200ca0a-93a5-4beb-9113-c4c711ee2ab6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:38.879653 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 812/1641] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:38 2022] PUT /identity/v3/projects/63e51fefc9ed4174826bff53fe6da705/users/38317a8501cc414cadb852610a8f69f2/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:38.881674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf256aa-ec46-4332-a0dd-cbe60fd7aac5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:38.882212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf256aa-ec46-4332-a0dd-cbe60fd7aac5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:38.883271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf256aa-ec46-4332-a0dd-cbe60fd7aac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:38.927742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4bf256aa-ec46-4332-a0dd-cbe60fd7aac5 None None] MFA Rules not processed for user `38317a8501cc414cadb852610a8f69f2`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:38.929191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4bf256aa-ec46-4332-a0dd-cbe60fd7aac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:39.044022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 830/1642] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:42:38 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:39.047715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200ca0a-93a5-4beb-9113-c4c711ee2ab6 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:39.066444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 813/1643] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:38 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:39.067461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf256aa-ec46-4332-a0dd-cbe60fd7aac5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:39.069743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:39.163881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:39.180241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf256aa-ec46-4332-a0dd-cbe60fd7aac5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:39.184464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4200ca0a-93a5-4beb-9113-c4c711ee2ab6 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:39.184926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004d976-8f67-4c90-bd15-19191bd5bd06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:39.185551 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004d976-8f67-4c90-bd15-19191bd5bd06 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:39.185832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004d976-8f67-4c90-bd15-19191bd5bd06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:39.186102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004d976-8f67-4c90-bd15-19191bd5bd06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:39.188585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc3bbed-2745-4af9-9ed1-42a08cd23012 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:39.189216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3bbed-2745-4af9-9ed1-42a08cd23012 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:39.189493 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3bbed-2745-4af9-9ed1-42a08cd23012 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:39.189831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc3bbed-2745-4af9-9ed1-42a08cd23012 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:39.197486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004d976-8f67-4c90-bd15-19191bd5bd06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:39.199594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004d976-8f67-4c90-bd15-19191bd5bd06 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:39.213414 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 814/1644] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:39.279626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3bbed-2745-4af9-9ed1-42a08cd23012 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:39.281474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc3bbed-2745-4af9-9ed1-42a08cd23012 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:39.308209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 831/1645] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:39 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:39.655880 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004d976-8f67-4c90-bd15-19191bd5bd06 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:39.665218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004d976-8f67-4c90-bd15-19191bd5bd06 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:39.674028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5af197-3752-44c7-9168-67b101da6bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:39.675191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5af197-3752-44c7-9168-67b101da6bbd service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:39.675503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5af197-3752-44c7-9168-67b101da6bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:39.677143 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5af197-3752-44c7-9168-67b101da6bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:39.825311 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5af197-3752-44c7-9168-67b101da6bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:39.827782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5af197-3752-44c7-9168-67b101da6bbd service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:39.842472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 815/1646] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:40.949639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c82d4-2f15-4e1b-b68a-e73e24a5577a None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:40.950970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c82d4-2f15-4e1b-b68a-e73e24a5577a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:40.951655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c82d4-2f15-4e1b-b68a-e73e24a5577a None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:40.956201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 832/1647] 10.222.0.39 () {54 vars in 962 bytes} [Tue Sep 6 22:42:40 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:42:40.982817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9becd-0edc-44cf-ac22-3dfe5c2c94e5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:40.983371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9becd-0edc-44cf-ac22-3dfe5c2c94e5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:40.983534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a9becd-0edc-44cf-ac22-3dfe5c2c94e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:41.049106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-75a9becd-0edc-44cf-ac22-3dfe5c2c94e5 None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:41.057560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-75a9becd-0edc-44cf-ac22-3dfe5c2c94e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:41.086424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 816/1648] 10.222.0.39 () {58 vars in 1059 bytes} [Tue Sep 6 22:42:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:42:41.095061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633a2ca-e905-4abe-af2d-f35046691558 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:41.095917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633a2ca-e905-4abe-af2d-f35046691558 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:41.096276 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3633a2ca-e905-4abe-af2d-f35046691558 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:41.098356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 833/1649] 10.222.0.39 () {54 vars in 912 bytes} [Tue Sep 6 22:42:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:42:41.104902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054648f-7dff-4386-b6c5-59ca05b7bbc3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:41.105181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054648f-7dff-4386-b6c5-59ca05b7bbc3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:41.105429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6054648f-7dff-4386-b6c5-59ca05b7bbc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:41.194026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6054648f-7dff-4386-b6c5-59ca05b7bbc3 None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:41.195313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6054648f-7dff-4386-b6c5-59ca05b7bbc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:41.238061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 817/1650] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:42:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:41.248039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633a2ca-e905-4abe-af2d-f35046691558 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:41.259981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d8c8d-bad8-43e9-90fc-f17798659a51 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:41.260103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d8c8d-bad8-43e9-90fc-f17798659a51 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:41.260285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d8c8d-bad8-43e9-90fc-f17798659a51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:41.275291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3633a2ca-e905-4abe-af2d-f35046691558 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:41.280018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3135c0-b8d7-4e90-af9b-37f56a96ec76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:41.280994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3135c0-b8d7-4e90-af9b-37f56a96ec76 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:41.281535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3135c0-b8d7-4e90-af9b-37f56a96ec76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:41.283223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3135c0-b8d7-4e90-af9b-37f56a96ec76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:41.300893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3135c0-b8d7-4e90-af9b-37f56a96ec76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:41.303728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3135c0-b8d7-4e90-af9b-37f56a96ec76 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:41.321877 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-776d8c8d-bad8-43e9-90fc-f17798659a51 None None] MFA Rules not processed for user `0daf3ccb6f554148b22d160e2bb77ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:41.324628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-776d8c8d-bad8-43e9-90fc-f17798659a51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:41.333888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 834/1651] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:41.480367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 818/1652] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:41 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:43.283537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81728d-6daa-4494-bb59-80369edadf1c None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:43.284131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81728d-6daa-4494-bb59-80369edadf1c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:43.284657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81728d-6daa-4494-bb59-80369edadf1c None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:43.286944 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 835/1653] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:42:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:42:43.335245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf62fa7-8807-4bca-85b0-d31c170db20a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:43.335245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf62fa7-8807-4bca-85b0-d31c170db20a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:43.335245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf62fa7-8807-4bca-85b0-d31c170db20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:43.371318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-eaf62fa7-8807-4bca-85b0-d31c170db20a None None] MFA Rules not processed for user `1d7e1b7dfa32448a88e0c90f10cdb2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:43.373369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-eaf62fa7-8807-4bca-85b0-d31c170db20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:43.482529 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:43.509867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 819/1654] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:42:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:43.685996 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81728d-6daa-4494-bb59-80369edadf1c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:43.723227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81728d-6daa-4494-bb59-80369edadf1c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:43.737480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94346d6b-8d61-45c8-b22d-e3ec569933d5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:43.738311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94346d6b-8d61-45c8-b22d-e3ec569933d5 service placement] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:43.738888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94346d6b-8d61-45c8-b22d-e3ec569933d5 service placement] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:43.739389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94346d6b-8d61-45c8-b22d-e3ec569933d5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:43.762034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94346d6b-8d61-45c8-b22d-e3ec569933d5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:43.772212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94346d6b-8d61-45c8-b22d-e3ec569933d5 service placement] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:43.790275 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 836/1655] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:45.060259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a29f8-d4a5-40d4-945a-447b2aa3d862 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:45.061312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a29f8-d4a5-40d4-945a-447b2aa3d862 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:45.061728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6a29f8-d4a5-40d4-945a-447b2aa3d862 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:45.063328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 820/1656] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:42:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:42:45.106962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fba1d-ba9b-4a1b-986e-daeb409d2ffb None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:45.107862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fba1d-ba9b-4a1b-986e-daeb409d2ffb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:45.108289 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30fba1d-ba9b-4a1b-986e-daeb409d2ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:45.148824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-a30fba1d-ba9b-4a1b-986e-daeb409d2ffb None None] MFA Rules not processed for user `1d7e1b7dfa32448a88e0c90f10cdb2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:45.150051 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a30fba1d-ba9b-4a1b-986e-daeb409d2ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:45.178280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 837/1657] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:42:45 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:45.192513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a29f8-d4a5-40d4-945a-447b2aa3d862 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:45.220105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6a29f8-d4a5-40d4-945a-447b2aa3d862 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:45.233231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3457f18-4ac7-4a83-a05f-d9cfe20eb196 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:45.233859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3457f18-4ac7-4a83-a05f-d9cfe20eb196 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:45.234458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3457f18-4ac7-4a83-a05f-d9cfe20eb196 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:45.235627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3457f18-4ac7-4a83-a05f-d9cfe20eb196 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:45.261715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3457f18-4ac7-4a83-a05f-d9cfe20eb196 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:45.263508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3457f18-4ac7-4a83-a05f-d9cfe20eb196 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:45.295318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 821/1658] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:45.687329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fba1d-ba9b-4a1b-986e-daeb409d2ffb None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:45.855454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30fba1d-ba9b-4a1b-986e-daeb409d2ffb None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:45.873423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:45.874211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:45.874659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:45.875045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] PATH_INFO: `/v3/users/ab043ae6f1964e4eba35ca45ef5e53cb` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:45.876415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab043ae6f1964e4eba35ca45ef5e53cb)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:45.877620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: WARNING py.warnings [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:42:45.877620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: warnings.warn(msg) Sep 06 22:42:45.877620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: Sep 06 22:42:45.879083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:45.997010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:46.069382 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab043ae6f1964e4eba35ca45ef5e53cb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:46.090590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab043ae6f1964e4eba35ca45ef5e53cb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:46.091340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab043ae6f1964e4eba35ca45ef5e53cb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:46.134291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab043ae6f1964e4eba35ca45ef5e53cb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:46.160498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab043ae6f1964e4eba35ca45ef5e53cb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:46.177724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 838/1659] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:42:45 2022] DELETE /identity/v3/users/ab043ae6f1964e4eba35ca45ef5e53cb => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:46.959561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3457f18-4ac7-4a83-a05f-d9cfe20eb196 service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:46.976171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3457f18-4ac7-4a83-a05f-d9cfe20eb196 service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:47.125493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:47.126147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:47.126421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:47.126687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] PATH_INFO: `/v3/projects/2ed669dd0269435a97ca38c4a869e50e` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:47.127848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ed669dd0269435a97ca38c4a869e50e)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:47.129067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: WARNING py.warnings [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 06 22:42:47.129067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: warnings.warn(msg) Sep 06 22:42:47.129067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: Sep 06 22:42:47.130992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:47.243432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ed669dd0269435a97ca38c4a869e50e', 'request_id': 'req-f8af116d-9c32-408c-8e24-d8f29b9869b1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:47.270849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ed669dd0269435a97ca38c4a869e50e', 'request_id': 'req-f8af116d-9c32-408c-8e24-d8f29b9869b1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:47.295169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ed669dd0269435a97ca38c4a869e50e', 'request_id': 'req-f8af116d-9c32-408c-8e24-d8f29b9869b1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:47.343743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] The token cache is being invalidate because project 2ed669dd0269435a97ca38c4a869e50e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:42:47.345067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:47.346988 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:47.349335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 822/1660] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:42:46 2022] DELETE /identity/v3/projects/2ed669dd0269435a97ca38c4a869e50e => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:47.353716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:47.357927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:47.407921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e5d7af-cfeb-4c64-83c0-88d6b93012fa admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:47.584449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:47.585158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:47.585478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:47.586555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] PATH_INFO: `/v3/users/0daf3ccb6f554148b22d160e2bb77ed8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:47.587975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0daf3ccb6f554148b22d160e2bb77ed8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:47.590110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:47.717913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0daf3ccb6f554148b22d160e2bb77ed8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:47.752850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0daf3ccb6f554148b22d160e2bb77ed8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:47.753709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0daf3ccb6f554148b22d160e2bb77ed8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:47.777838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0daf3ccb6f554148b22d160e2bb77ed8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:47.799050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0daf3ccb6f554148b22d160e2bb77ed8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:47.819693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 839/1661] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:42:47 2022] DELETE /identity/v3/users/0daf3ccb6f554148b22d160e2bb77ed8 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:48.574298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:48.576658 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:48.695561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af116d-9c32-408c-8e24-d8f29b9869b1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:48.790806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:48.886141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef54854-af31-4c69-937b-196f573b544d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:48.886964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef54854-af31-4c69-937b-196f573b544d service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:48.887347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef54854-af31-4c69-937b-196f573b544d service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:48.887547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef54854-af31-4c69-937b-196f573b544d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:48.893362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4ef54854-af31-4c69-937b-196f573b544d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:49.022078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:49.023699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:49.137718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef54854-af31-4c69-937b-196f573b544d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:49.139188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef54854-af31-4c69-937b-196f573b544d service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:49.141854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:49.166969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 823/1662] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:48 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:42:49.246358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf420d32-7f99-4a87-90ae-1e55add545c4 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:49.263575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:49.264154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:49.264419 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:49.265085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] PATH_INFO: `/v3/projects/492dbc3ce2a64ad48f7596bad133ab25` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:49.269635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=492dbc3ce2a64ad48f7596bad133ab25)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:49.271756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:49.422968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '492dbc3ce2a64ad48f7596bad133ab25', 'request_id': 'req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:49.439989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '492dbc3ce2a64ad48f7596bad133ab25', 'request_id': 'req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:49.447112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '492dbc3ce2a64ad48f7596bad133ab25', 'request_id': 'req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:49.478963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] The token cache is being invalidate because project 492dbc3ce2a64ad48f7596bad133ab25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:42:49.479258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:49.479937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1b5ba6-95ca-4a47-bb96-c9563404bda6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:49.481798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 840/1663] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:42:49 2022] DELETE /identity/v3/projects/492dbc3ce2a64ad48f7596bad133ab25 => generated 0 bytes in 460 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:49.495920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8f03d-c38d-4557-bfca-d75310d28ee1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:49.496123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8f03d-c38d-4557-bfca-d75310d28ee1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:49.496398 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a8f03d-c38d-4557-bfca-d75310d28ee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:49.539646 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-93a8f03d-c38d-4557-bfca-d75310d28ee1 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:49.540933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-93a8f03d-c38d-4557-bfca-d75310d28ee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:49.676292 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 824/1664] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:49.678068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf24e9-feec-4372-a46d-930e3f2ed8c4 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:49.678340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf24e9-feec-4372-a46d-930e3f2ed8c4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:49.678608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf24e9-feec-4372-a46d-930e3f2ed8c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:49.717283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-9caf24e9-feec-4372-a46d-930e3f2ed8c4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:49.718529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9caf24e9-feec-4372-a46d-930e3f2ed8c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:49.740548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 841/1665] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:49.741828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8f03d-c38d-4557-bfca-d75310d28ee1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:49.755988 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a8f03d-c38d-4557-bfca-d75310d28ee1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:49.780319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc14a9-e930-426c-9822-65d4ce77d93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:49.780977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc14a9-e930-426c-9822-65d4ce77d93f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:49.781253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc14a9-e930-426c-9822-65d4ce77d93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:49.782102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45fc14a9-e930-426c-9822-65d4ce77d93f admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:49.782762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc14a9-e930-426c-9822-65d4ce77d93f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:42:49.783045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc14a9-e930-426c-9822-65d4ce77d93f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:49.785287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45fc14a9-e930-426c-9822-65d4ce77d93f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:49.816003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 825/1666] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:42:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:42:49.820347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf24e9-feec-4372-a46d-930e3f2ed8c4 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:49.840752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf24e9-feec-4372-a46d-930e3f2ed8c4 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:49.853614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b1182-7cd4-4d67-b96c-3b7b2ded5598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:49.854251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b1182-7cd4-4d67-b96c-3b7b2ded5598 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:49.854529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b1182-7cd4-4d67-b96c-3b7b2ded5598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:49.854795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301b1182-7cd4-4d67-b96c-3b7b2ded5598 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:49.855436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b1182-7cd4-4d67-b96c-3b7b2ded5598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:49.857395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301b1182-7cd4-4d67-b96c-3b7b2ded5598 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:49.916713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 842/1667] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:49 2022] POST /identity/v3/projects => generated 353 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:49.921032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc14a9-e930-426c-9822-65d4ce77d93f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:49.940494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45fc14a9-e930-426c-9822-65d4ce77d93f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:50.041685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d2145-97e4-44bc-b4b1-8ed0a195be48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:50.042466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d2145-97e4-44bc-b4b1-8ed0a195be48 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:50.042860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d2145-97e4-44bc-b4b1-8ed0a195be48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:50.043242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3d2145-97e4-44bc-b4b1-8ed0a195be48 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:50.044003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d2145-97e4-44bc-b4b1-8ed0a195be48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:50.046166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3d2145-97e4-44bc-b4b1-8ed0a195be48 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:50.073766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:50.109959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 826/1668] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:49 2022] POST /identity/v3/users => generated 277 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:50.112473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b1182-7cd4-4d67-b96c-3b7b2ded5598 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:50.126915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301b1182-7cd4-4d67-b96c-3b7b2ded5598 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:50.136888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28bf20-30d7-4e80-94fe-6b39b4aa2ea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:50.138203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28bf20-30d7-4e80-94fe-6b39b4aa2ea0 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:50.138810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28bf20-30d7-4e80-94fe-6b39b4aa2ea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:50.139476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae28bf20-30d7-4e80-94fe-6b39b4aa2ea0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:50.141753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28bf20-30d7-4e80-94fe-6b39b4aa2ea0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:50.144154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae28bf20-30d7-4e80-94fe-6b39b4aa2ea0 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:50.160431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 843/1669] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:50.165621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d2145-97e4-44bc-b4b1-8ed0a195be48 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:50.182703 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3d2145-97e4-44bc-b4b1-8ed0a195be48 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:50.197526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3044bc60-9d19-4fa9-bced-fb645652567a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:50.198546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044bc60-9d19-4fa9-bced-fb645652567a admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:50.199040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044bc60-9d19-4fa9-bced-fb645652567a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:50.199583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3044bc60-9d19-4fa9-bced-fb645652567a admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:50.200456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044bc60-9d19-4fa9-bced-fb645652567a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:50.205147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3044bc60-9d19-4fa9-bced-fb645652567a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:50.243193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 827/1670] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:50 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:50.246710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28bf20-30d7-4e80-94fe-6b39b4aa2ea0 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:50.266783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae28bf20-30d7-4e80-94fe-6b39b4aa2ea0 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:50.283059 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d75e0-312d-4642-a504-6841f4846b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:50.284374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d75e0-312d-4642-a504-6841f4846b9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:50.286357 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d75e0-312d-4642-a504-6841f4846b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:50.286740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2d75e0-312d-4642-a504-6841f4846b9d admin admin] PATH_INFO: `/v3/projects/61ac98bf75ff4ee4ab6bc3ac6540e385/users/48d26e4ad5914e8f9fb97109002ef80f/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:50.367581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d75e0-312d-4642-a504-6841f4846b9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=61ac98bf75ff4ee4ab6bc3ac6540e385, user_id=48d26e4ad5914e8f9fb97109002ef80f, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:50.374591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2d75e0-312d-4642-a504-6841f4846b9d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:50.405311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 844/1671] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:50 2022] PUT /identity/v3/projects/61ac98bf75ff4ee4ab6bc3ac6540e385/users/48d26e4ad5914e8f9fb97109002ef80f/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:50.415826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1c5ff-d97d-4ec1-a203-7049684fd401 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:50.416114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1c5ff-d97d-4ec1-a203-7049684fd401 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:50.416405 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac1c5ff-d97d-4ec1-a203-7049684fd401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:50.501185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1ac1c5ff-d97d-4ec1-a203-7049684fd401 None None] MFA Rules not processed for user `48d26e4ad5914e8f9fb97109002ef80f`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:50.505087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1ac1c5ff-d97d-4ec1-a203-7049684fd401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:50.738176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 828/1672] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:50 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:50.742855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d75e0-312d-4642-a504-6841f4846b9d admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:50.858206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:50.907647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a7877-9010-4ce0-854a-0573f4227b77 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:50.908143 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a7877-9010-4ce0-854a-0573f4227b77 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:50.908579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08a7877-9010-4ce0-854a-0573f4227b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:50.910882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2d75e0-312d-4642-a504-6841f4846b9d admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:50.924656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08918096-06ae-4a22-a4a8-494931af9c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:50.925416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08918096-06ae-4a22-a4a8-494931af9c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:50.925779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08918096-06ae-4a22-a4a8-494931af9c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:50.926855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08918096-06ae-4a22-a4a8-494931af9c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:50.928152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08918096-06ae-4a22-a4a8-494931af9c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:50.930324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08918096-06ae-4a22-a4a8-494931af9c75 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:50.970703 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b08a7877-9010-4ce0-854a-0573f4227b77 None None] MFA Rules not processed for user `4dd9f0bff3044d7db561dff21ac0e7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:50.972467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b08a7877-9010-4ce0-854a-0573f4227b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:51.027676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 845/1673] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:50 2022] POST /identity/v3/projects => generated 357 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:51.034053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08918096-06ae-4a22-a4a8-494931af9c75 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:51.130399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 829/1674] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:51.143329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a7877-9010-4ce0-854a-0573f4227b77 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:51.145203 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b08a7877-9010-4ce0-854a-0573f4227b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:51.203875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08918096-06ae-4a22-a4a8-494931af9c75 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:51.218853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf65bc8-eb50-4db0-a43f-379375b45235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:51.219561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf65bc8-eb50-4db0-a43f-379375b45235 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:51.220028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf65bc8-eb50-4db0-a43f-379375b45235 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:51.220214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf65bc8-eb50-4db0-a43f-379375b45235 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:51.220888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf65bc8-eb50-4db0-a43f-379375b45235 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:51.222822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf65bc8-eb50-4db0-a43f-379375b45235 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:51.276250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 846/1675] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:51 2022] POST /identity/v3/users => generated 285 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:51.277804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c523e-10b4-42c0-945b-e3b76325c38c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:51.278308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c523e-10b4-42c0-945b-e3b76325c38c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:51.278724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6c523e-10b4-42c0-945b-e3b76325c38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:51.338339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:51.346731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-4a6c523e-10b4-42c0-945b-e3b76325c38c None None] MFA Rules not processed for user `2945d87f1c114dd0b42c4e58a8133e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:51.348461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4a6c523e-10b4-42c0-945b-e3b76325c38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:51.389732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08a7877-9010-4ce0-854a-0573f4227b77 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:51.522088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 847/1676] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:51.523584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c523e-10b4-42c0-945b-e3b76325c38c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:51.557999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6c523e-10b4-42c0-945b-e3b76325c38c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:51.577456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b379df17-3d5d-4e82-a9b2-484bd432e058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:51.579147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379df17-3d5d-4e82-a9b2-484bd432e058 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:51.579675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379df17-3d5d-4e82-a9b2-484bd432e058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:51.581069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b379df17-3d5d-4e82-a9b2-484bd432e058 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:51.581837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379df17-3d5d-4e82-a9b2-484bd432e058 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:51.585885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b379df17-3d5d-4e82-a9b2-484bd432e058 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:51.602259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37432a1-7f07-4a76-8841-99d20917f9e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:51.603037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37432a1-7f07-4a76-8841-99d20917f9e5 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:51.603402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37432a1-7f07-4a76-8841-99d20917f9e5 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:51.603736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37432a1-7f07-4a76-8841-99d20917f9e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:51.612431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 848/1677] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:51.617889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b41fc6-2470-4a77-93fa-031dd3469b64 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:51.617889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b41fc6-2470-4a77-93fa-031dd3469b64 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:51.617889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b41fc6-2470-4a77-93fa-031dd3469b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:51.677129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-04b41fc6-2470-4a77-93fa-031dd3469b64 None None] MFA Rules not processed for user `2945d87f1c114dd0b42c4e58a8133e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:51.679628 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-04b41fc6-2470-4a77-93fa-031dd3469b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:51.703020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 849/1678] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:51.704542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b41fc6-2470-4a77-93fa-031dd3469b64 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:51.729654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b41fc6-2470-4a77-93fa-031dd3469b64 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:51.743052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37432a1-7f07-4a76-8841-99d20917f9e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:51.745350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37432a1-7f07-4a76-8841-99d20917f9e5 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:51.749535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28c999-8f16-48b3-be97-27f781d2c6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:51.750893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28c999-8f16-48b3-be97-27f781d2c6ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:51.751288 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28c999-8f16-48b3-be97-27f781d2c6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:51.751565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de28c999-8f16-48b3-be97-27f781d2c6ec admin admin] PATH_INFO: `/v3/projects/176f0d6a7c264d93a85139232dad3e8e/users/aacee78ca3fb48af96b27e1f44880bdf/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:51.757269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 830/1679] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:42:51.758154 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37432a1-7f07-4a76-8841-99d20917f9e5 service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:51.759600 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c37432a1-7f07-4a76-8841-99d20917f9e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:51.786562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37432a1-7f07-4a76-8841-99d20917f9e5 service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:51.804178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6231eed-7c96-40ee-ad7e-067e1988e557 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:51.805126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6231eed-7c96-40ee-ad7e-067e1988e557 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:51.805534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6231eed-7c96-40ee-ad7e-067e1988e557 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:51.805876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6231eed-7c96-40ee-ad7e-067e1988e557 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:51.833904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28c999-8f16-48b3-be97-27f781d2c6ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=176f0d6a7c264d93a85139232dad3e8e, user_id=aacee78ca3fb48af96b27e1f44880bdf, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:51.835885 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6231eed-7c96-40ee-ad7e-067e1988e557 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:51.837886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6231eed-7c96-40ee-ad7e-067e1988e557 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:51.841211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de28c999-8f16-48b3-be97-27f781d2c6ec admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:51.875956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 831/1680] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:51 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:51.910630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 850/1681] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:51 2022] PUT /identity/v3/projects/176f0d6a7c264d93a85139232dad3e8e/users/aacee78ca3fb48af96b27e1f44880bdf/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:51.911875 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6231eed-7c96-40ee-ad7e-067e1988e557 service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:51.935537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6231eed-7c96-40ee-ad7e-067e1988e557 service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:52.088336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1551d2-88d1-4dc2-bfb3-caab34e56711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:52.089293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1551d2-88d1-4dc2-bfb3-caab34e56711 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:52.089896 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1551d2-88d1-4dc2-bfb3-caab34e56711 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:52.090346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1551d2-88d1-4dc2-bfb3-caab34e56711 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:52.091218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1551d2-88d1-4dc2-bfb3-caab34e56711 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:52.097795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1551d2-88d1-4dc2-bfb3-caab34e56711 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:52.111863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 832/1682] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:51 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:42:52.116429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28c999-8f16-48b3-be97-27f781d2c6ec admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:52.131840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de28c999-8f16-48b3-be97-27f781d2c6ec admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:52.189642 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57de21-3b28-40ad-8dac-20e882fe26bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:52.190356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57de21-3b28-40ad-8dac-20e882fe26bb admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:52.190635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57de21-3b28-40ad-8dac-20e882fe26bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:52.190915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57de21-3b28-40ad-8dac-20e882fe26bb admin admin] PATH_INFO: `/v3/projects/176f0d6a7c264d93a85139232dad3e8e/users/aacee78ca3fb48af96b27e1f44880bdf/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:52.199530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57de21-3b28-40ad-8dac-20e882fe26bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=176f0d6a7c264d93a85139232dad3e8e, user_id=aacee78ca3fb48af96b27e1f44880bdf, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:52.208416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57de21-3b28-40ad-8dac-20e882fe26bb admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:52.228503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:52.259615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 851/1683] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:42:52 2022] PUT /identity/v3/projects/176f0d6a7c264d93a85139232dad3e8e/users/aacee78ca3fb48af96b27e1f44880bdf/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:52.260663 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98380186-8c67-46f6-9d08-80943e020459 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:52.260995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98380186-8c67-46f6-9d08-80943e020459 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:52.261321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98380186-8c67-46f6-9d08-80943e020459 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:52.386671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-98380186-8c67-46f6-9d08-80943e020459 None None] MFA Rules not processed for user `aacee78ca3fb48af96b27e1f44880bdf`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:52.389739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-98380186-8c67-46f6-9d08-80943e020459 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:52.423895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:52.676114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 833/1684] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:52 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:52.689822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57de21-3b28-40ad-8dac-20e882fe26bb admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:52.692011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8c57de21-3b28-40ad-8dac-20e882fe26bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:52.832114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57de21-3b28-40ad-8dac-20e882fe26bb admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:52.839452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d350e55e-da12-4eaa-b131-bf5ea4b7caf7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:52.840092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350e55e-da12-4eaa-b131-bf5ea4b7caf7 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:52.840365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350e55e-da12-4eaa-b131-bf5ea4b7caf7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:52.840636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350e55e-da12-4eaa-b131-bf5ea4b7caf7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:52.866837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350e55e-da12-4eaa-b131-bf5ea4b7caf7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:52.869076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d350e55e-da12-4eaa-b131-bf5ea4b7caf7 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:52.909867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 852/1685] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:52 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:53.194163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98380186-8c67-46f6-9d08-80943e020459 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:53.222087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98380186-8c67-46f6-9d08-80943e020459 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:53.244322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a074401-1f61-4106-94fa-fe045ce9e488 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:53.245155 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a074401-1f61-4106-94fa-fe045ce9e488 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:53.245575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a074401-1f61-4106-94fa-fe045ce9e488 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:53.247147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 853/1686] 10.222.0.39 () {54 vars in 912 bytes} [Tue Sep 6 22:42:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 06 22:42:53.247297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ba8de5-9b03-44be-ac6f-bb6cfb9069d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:53.249162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba8de5-9b03-44be-ac6f-bb6cfb9069d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:53.249441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba8de5-9b03-44be-ac6f-bb6cfb9069d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:53.251113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ba8de5-9b03-44be-ac6f-bb6cfb9069d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:53.258315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636d745-848f-427f-843d-05b99481626e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:53.258919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636d745-848f-427f-843d-05b99481626e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:53.259121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636d745-848f-427f-843d-05b99481626e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:53.380257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2636d745-848f-427f-843d-05b99481626e None None] MFA Rules not processed for user `2658435f74a947f39e207465ebff22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:53.383364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2636d745-848f-427f-843d-05b99481626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:53.544827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba8de5-9b03-44be-ac6f-bb6cfb9069d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:53.548217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ba8de5-9b03-44be-ac6f-bb6cfb9069d7 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:53.568874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 854/1687] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:42:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 06 22:42:53.594689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 834/1688] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:53 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:53.610472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636d745-848f-427f-843d-05b99481626e None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:53.612334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2636d745-848f-427f-843d-05b99481626e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:53.650503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2636d745-848f-427f-843d-05b99481626e None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:53.670044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9174b-43b7-4231-893b-1c1bc0dd1eae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:53.670784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9174b-43b7-4231-893b-1c1bc0dd1eae service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:53.671133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9174b-43b7-4231-893b-1c1bc0dd1eae service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:53.671492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9174b-43b7-4231-893b-1c1bc0dd1eae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:53.707682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9174b-43b7-4231-893b-1c1bc0dd1eae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:53.711627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9174b-43b7-4231-893b-1c1bc0dd1eae service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:53.736149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 855/1689] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:54.621006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d657d3-5f91-4282-8822-8c6865cb8ab0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:54.621536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d657d3-5f91-4282-8822-8c6865cb8ab0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:54.621536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d657d3-5f91-4282-8822-8c6865cb8ab0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:54.715262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-67d657d3-5f91-4282-8822-8c6865cb8ab0 None None] MFA Rules not processed for user `da5264060ddd48afa8305c38a9bfa4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:54.716506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-67d657d3-5f91-4282-8822-8c6865cb8ab0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:54.872017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9174b-43b7-4231-893b-1c1bc0dd1eae service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:54.873401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-cac9174b-43b7-4231-893b-1c1bc0dd1eae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:54.982625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 835/1690] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:54.994246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c564c4-5177-4542-915f-968f462e1940 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:54.994742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c564c4-5177-4542-915f-968f462e1940 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:54.995315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c564c4-5177-4542-915f-968f462e1940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:55.039493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-71c564c4-5177-4542-915f-968f462e1940 None None] MFA Rules not processed for user `da5264060ddd48afa8305c38a9bfa4c3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:55.042655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-71c564c4-5177-4542-915f-968f462e1940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:55.079108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 836/1691] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:42:55.103117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0439648-9c92-471a-8b3c-613985510946 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:55.103491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0439648-9c92-471a-8b3c-613985510946 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:55.103886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0439648-9c92-471a-8b3c-613985510946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:55.162853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f0439648-9c92-471a-8b3c-613985510946 None None] MFA Rules not processed for user `2945d87f1c114dd0b42c4e58a8133e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:55.166671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f0439648-9c92-471a-8b3c-613985510946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:55.185487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9174b-43b7-4231-893b-1c1bc0dd1eae service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:55.214911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf613cc-2b79-42ef-b455-8c263fdef31f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:55.215598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf613cc-2b79-42ef-b455-8c263fdef31f service placement] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:55.215963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf613cc-2b79-42ef-b455-8c263fdef31f service placement] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:55.216896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf613cc-2b79-42ef-b455-8c263fdef31f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:55.225054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ddf613cc-2b79-42ef-b455-8c263fdef31f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:55.227265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:55.294034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf613cc-2b79-42ef-b455-8c263fdef31f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:55.298464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf613cc-2b79-42ef-b455-8c263fdef31f service placement] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:55.332089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 856/1692] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 461 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:42:55.336239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf613cc-2b79-42ef-b455-8c263fdef31f service placement] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:55.383014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 837/1693] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:55 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:42:55.406798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0439648-9c92-471a-8b3c-613985510946 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:55.410074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f0439648-9c92-471a-8b3c-613985510946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:55.425329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:55.720946 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf613cc-2b79-42ef-b455-8c263fdef31f service placement] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:55.773129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6eeb47-d0e7-451c-8b29-4a50825150e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:55.776449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0439648-9c92-471a-8b3c-613985510946 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:55.777566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eeb47-d0e7-451c-8b29-4a50825150e1 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:55.778546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eeb47-d0e7-451c-8b29-4a50825150e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:55.778546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6eeb47-d0e7-451c-8b29-4a50825150e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:55.793907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:55.798767 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:55.799121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:55.799481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] PATH_INFO: `/v3/users/a742445cb56f4e1b951e76acfc16a695` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:55.801092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a742445cb56f4e1b951e76acfc16a695)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:55.806670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:55.808498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6eeb47-d0e7-451c-8b29-4a50825150e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:55.810416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6eeb47-d0e7-451c-8b29-4a50825150e1 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:55.858831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 857/1694] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 523 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:56.027864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a742445cb56f4e1b951e76acfc16a695'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:56.030852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a742445cb56f4e1b951e76acfc16a695'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:56.055246 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a742445cb56f4e1b951e76acfc16a695'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:56.073444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a742445cb56f4e1b951e76acfc16a695'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:56.157789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a742445cb56f4e1b951e76acfc16a695'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:56.182034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 838/1695] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:42:55 2022] DELETE /identity/v3/users/a742445cb56f4e1b951e76acfc16a695 => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:56.183782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b396c9-5d72-40a0-9953-5bc3e784b015 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:56.184584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b396c9-5d72-40a0-9953-5bc3e784b015 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:56.185110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b396c9-5d72-40a0-9953-5bc3e784b015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:56.237294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-f6b396c9-5d72-40a0-9953-5bc3e784b015 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:56.238546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f6b396c9-5d72-40a0-9953-5bc3e784b015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:56.405742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 858/1696] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:56.430621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:56.432497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:56.630168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd62a34-7db7-4354-91ca-d3e6358a215b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:56.648163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b70a20-e443-4843-af86-607b4b5feb46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:56.648606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b70a20-e443-4843-af86-607b4b5feb46 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:56.648923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b70a20-e443-4843-af86-607b4b5feb46 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:56.649187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b70a20-e443-4843-af86-607b4b5feb46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:56.676003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:56.684247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b70a20-e443-4843-af86-607b4b5feb46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:56.686655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b70a20-e443-4843-af86-607b4b5feb46 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:56.701959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 839/1697] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:57.043253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181bf8a-082b-4ba8-bcb7-0b33802160f7 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:57.043548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181bf8a-082b-4ba8-bcb7-0b33802160f7 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:57.043978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5181bf8a-082b-4ba8-bcb7-0b33802160f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:57.150747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-5181bf8a-082b-4ba8-bcb7-0b33802160f7 None None] MFA Rules not processed for user `e305e7d0cbfc4aeca6164685d0155475`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:57.152161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5181bf8a-082b-4ba8-bcb7-0b33802160f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:57.302786 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145487c3-de34-4225-ba55-c2e664febf14 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:57.304036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145487c3-de34-4225-ba55-c2e664febf14 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:57.304467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145487c3-de34-4225-ba55-c2e664febf14 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:57.306417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 840/1698] 10.222.0.39 () {54 vars in 912 bytes} [Tue Sep 6 22:42:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:42:57.313365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a574e5-52cf-4c41-9fee-95a8b1a8ab43 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:57.313689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a574e5-52cf-4c41-9fee-95a8b1a8ab43 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:57.313991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a574e5-52cf-4c41-9fee-95a8b1a8ab43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:57.333219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 859/1699] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:57.372501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-26a574e5-52cf-4c41-9fee-95a8b1a8ab43 None None] MFA Rules not processed for user `2658435f74a947f39e207465ebff22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:57.383289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-26a574e5-52cf-4c41-9fee-95a8b1a8ab43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:57.400178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 841/1700] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:42:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:42:57.675741 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181bf8a-082b-4ba8-bcb7-0b33802160f7 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:57.702923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5181bf8a-082b-4ba8-bcb7-0b33802160f7 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:57.722825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ee7f0-e441-447d-9474-6bca473f6081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:57.723547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ee7f0-e441-447d-9474-6bca473f6081 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:57.723827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ee7f0-e441-447d-9474-6bca473f6081 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:57.724168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2ee7f0-e441-447d-9474-6bca473f6081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:57.746196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ee7f0-e441-447d-9474-6bca473f6081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:57.748937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2ee7f0-e441-447d-9474-6bca473f6081 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:57.799260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 860/1701] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:57.925159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a574e5-52cf-4c41-9fee-95a8b1a8ab43 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:57.926816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-26a574e5-52cf-4c41-9fee-95a8b1a8ab43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:58.157294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a574e5-52cf-4c41-9fee-95a8b1a8ab43 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:58.170824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421b5ad4-838b-405b-b31f-55014ecf597f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:58.173426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b5ad4-838b-405b-b31f-55014ecf597f service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:58.174081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b5ad4-838b-405b-b31f-55014ecf597f service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:58.174581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421b5ad4-838b-405b-b31f-55014ecf597f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:58.214103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b5ad4-838b-405b-b31f-55014ecf597f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:58.220116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421b5ad4-838b-405b-b31f-55014ecf597f service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:58.257550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 842/1702] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:42:57 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:42:58.606971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ee7f0-e441-447d-9474-6bca473f6081 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:58.610671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7d2ee7f0-e441-447d-9474-6bca473f6081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:58.637194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2ee7f0-e441-447d-9474-6bca473f6081 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:58.645920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:58.647288 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:58.647801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:58.649284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] PATH_INFO: `/v3/projects/200607d5d1f8491e9168dddd9903b779` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:58.652647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=200607d5d1f8491e9168dddd9903b779)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:58.658126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:58.686858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:58.791306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '200607d5d1f8491e9168dddd9903b779', 'request_id': 'req-35bccaa1-0cfe-48af-beeb-264be6cde4f6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:58.816358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '200607d5d1f8491e9168dddd9903b779', 'request_id': 'req-35bccaa1-0cfe-48af-beeb-264be6cde4f6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:58.840448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '200607d5d1f8491e9168dddd9903b779', 'request_id': 'req-35bccaa1-0cfe-48af-beeb-264be6cde4f6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:58.860862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] The token cache is being invalidate because project 200607d5d1f8491e9168dddd9903b779 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:42:58.861480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:58.863898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-35bccaa1-0cfe-48af-beeb-264be6cde4f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:42:58.866917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 861/1703] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:42:58 2022] DELETE /identity/v3/projects/200607d5d1f8491e9168dddd9903b779 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:42:58.872138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a200df1-d157-46a5-99e0-dc10a9f3ce9a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:58.873291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a200df1-d157-46a5-99e0-dc10a9f3ce9a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:58.874113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a200df1-d157-46a5-99e0-dc10a9f3ce9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:58.931106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2a200df1-d157-46a5-99e0-dc10a9f3ce9a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:58.946056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2a200df1-d157-46a5-99e0-dc10a9f3ce9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:59.178377 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12c339-9f37-48e4-9b77-a36f75ddcaa7 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:59.178925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12c339-9f37-48e4-9b77-a36f75ddcaa7 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:59.179689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c12c339-9f37-48e4-9b77-a36f75ddcaa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:59.205553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 843/1704] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:59.213017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb91f-b3a5-41a1-80d4-3a17503b8307 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:59.213295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb91f-b3a5-41a1-80d4-3a17503b8307 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:59.213748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cfb91f-b3a5-41a1-80d4-3a17503b8307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:59.242173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7c12c339-9f37-48e4-9b77-a36f75ddcaa7 None None] MFA Rules not processed for user `4dd9f0bff3044d7db561dff21ac0e7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:59.243830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7c12c339-9f37-48e4-9b77-a36f75ddcaa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:59.287081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f9cfb91f-b3a5-41a1-80d4-3a17503b8307 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:42:59.288390 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f9cfb91f-b3a5-41a1-80d4-3a17503b8307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:42:59.318729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 844/1705] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:42:59.322618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfb91f-b3a5-41a1-80d4-3a17503b8307 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:59.344594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cfb91f-b3a5-41a1-80d4-3a17503b8307 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:59.354740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 862/1706] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:42:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:42:59.392883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04787916-19cb-4b2d-bdb6-c279d36ccaf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:59.393780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04787916-19cb-4b2d-bdb6-c279d36ccaf7 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:59.394060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04787916-19cb-4b2d-bdb6-c279d36ccaf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:59.394327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04787916-19cb-4b2d-bdb6-c279d36ccaf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:59.394982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04787916-19cb-4b2d-bdb6-c279d36ccaf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:42:59.395275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04787916-19cb-4b2d-bdb6-c279d36ccaf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:59.396887 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04787916-19cb-4b2d-bdb6-c279d36ccaf7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:59.446532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 845/1707] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:42:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:42:59.456394 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12c339-9f37-48e4-9b77-a36f75ddcaa7 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:59.510551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c12c339-9f37-48e4-9b77-a36f75ddcaa7 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:59.523115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c543c1-1c93-4bf3-90cd-f16201f44ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:59.523761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c543c1-1c93-4bf3-90cd-f16201f44ca7 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:59.524217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c543c1-1c93-4bf3-90cd-f16201f44ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:59.524666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c543c1-1c93-4bf3-90cd-f16201f44ca7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:59.525466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c543c1-1c93-4bf3-90cd-f16201f44ca7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:59.527596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c543c1-1c93-4bf3-90cd-f16201f44ca7 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:59.619368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 863/1708] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:42:59 2022] POST /identity/v3/projects => generated 403 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:59.623335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04787916-19cb-4b2d-bdb6-c279d36ccaf7 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:59.638214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04787916-19cb-4b2d-bdb6-c279d36ccaf7 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:59.808315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:42:59.856869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f9f2f-1238-4b5d-9158-08d669076bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:59.857526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f9f2f-1238-4b5d-9158-08d669076bdd admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:42:59.857808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f9f2f-1238-4b5d-9158-08d669076bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:42:59.858083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5f9f2f-1238-4b5d-9158-08d669076bdd admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:42:59.859103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f9f2f-1238-4b5d-9158-08d669076bdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:42:59.870838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5f9f2f-1238-4b5d-9158-08d669076bdd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:42:59.967003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c543c1-1c93-4bf3-90cd-f16201f44ca7 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:42:59.975890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 846/1709] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:42:59 2022] POST /identity/v3/users => generated 302 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:42:59.985296 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c543c1-1c93-4bf3-90cd-f16201f44ca7 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:42:59.998872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bf55a-ffac-4c54-bd42-2c27391a3532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:42:59.999970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf55a-ffac-4c54-bd42-2c27391a3532 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:00.000301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf55a-ffac-4c54-bd42-2c27391a3532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:00.019046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7bf55a-ffac-4c54-bd42-2c27391a3532 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:00.020256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bf55a-ffac-4c54-bd42-2c27391a3532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:00.024312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7bf55a-ffac-4c54-bd42-2c27391a3532 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:00.076105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 864/1710] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:42:59 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:43:00.078986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f9f2f-1238-4b5d-9158-08d669076bdd admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:00.092710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5f9f2f-1238-4b5d-9158-08d669076bdd admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:00.107616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562d425-8e3f-4e2d-bc89-37353c7e32c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:00.108990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562d425-8e3f-4e2d-bc89-37353c7e32c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:00.109498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562d425-8e3f-4e2d-bc89-37353c7e32c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:00.110478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562d425-8e3f-4e2d-bc89-37353c7e32c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:00.113555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562d425-8e3f-4e2d-bc89-37353c7e32c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:00.116831 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562d425-8e3f-4e2d-bc89-37353c7e32c9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:00.146120 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bf55a-ffac-4c54-bd42-2c27391a3532 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:00.161861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 847/1711] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:00 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:00.166022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7bf55a-ffac-4c54-bd42-2c27391a3532 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:00.174726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a020f2c-e360-4958-af06-d812e0087484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:00.175754 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a020f2c-e360-4958-af06-d812e0087484 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:00.176245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a020f2c-e360-4958-af06-d812e0087484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:00.176711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a020f2c-e360-4958-af06-d812e0087484 admin admin] PATH_INFO: `/v3/projects/4f72d0727933497c8e805854bd226db8/users/8d5ffaecc5674700b4b5f12ed3938122/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:00.196986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a020f2c-e360-4958-af06-d812e0087484 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f72d0727933497c8e805854bd226db8, user_id=8d5ffaecc5674700b4b5f12ed3938122, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:00.200165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a020f2c-e360-4958-af06-d812e0087484 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:00.221547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 865/1712] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:00 2022] PUT /identity/v3/projects/4f72d0727933497c8e805854bd226db8/users/8d5ffaecc5674700b4b5f12ed3938122/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:00.242627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b702c86-2390-43a4-b9f8-df146e9ad12f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:00.244850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b702c86-2390-43a4-b9f8-df146e9ad12f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:00.245568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b702c86-2390-43a4-b9f8-df146e9ad12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:00.369770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4b702c86-2390-43a4-b9f8-df146e9ad12f None None] MFA Rules not processed for user `8d5ffaecc5674700b4b5f12ed3938122`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:00.377122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4b702c86-2390-43a4-b9f8-df146e9ad12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:00.582893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 848/1713] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:00 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 344 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:00.583598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a020f2c-e360-4958-af06-d812e0087484 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:00.713548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:00.757505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a020f2c-e360-4958-af06-d812e0087484 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:00.763012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13629b-27cb-4996-953a-116f225c8e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:00.763956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13629b-27cb-4996-953a-116f225c8e4b admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:00.764442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13629b-27cb-4996-953a-116f225c8e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:00.764947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe13629b-27cb-4996-953a-116f225c8e4b admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:00.766352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13629b-27cb-4996-953a-116f225c8e4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:00.769522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe13629b-27cb-4996-953a-116f225c8e4b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:00.847123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 866/1714] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:00 2022] POST /identity/v3/projects => generated 403 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:00.851163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b702c86-2390-43a4-b9f8-df146e9ad12f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:00.868399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b702c86-2390-43a4-b9f8-df146e9ad12f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:01.024415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6506f97-9002-41f6-b3ad-f6169925c172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:01.025127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6506f97-9002-41f6-b3ad-f6169925c172 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:01.025410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6506f97-9002-41f6-b3ad-f6169925c172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:01.025676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6506f97-9002-41f6-b3ad-f6169925c172 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:01.026312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6506f97-9002-41f6-b3ad-f6169925c172 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:01.032525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6506f97-9002-41f6-b3ad-f6169925c172 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:01.095723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 849/1715] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:00 2022] POST /identity/v3/users => generated 308 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:01.096987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13629b-27cb-4996-953a-116f225c8e4b admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:01.113646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe13629b-27cb-4996-953a-116f225c8e4b admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:01.128909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd91d0-d3c6-4a1f-a034-9259c41202d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:01.129802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd91d0-d3c6-4a1f-a034-9259c41202d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:01.130883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd91d0-d3c6-4a1f-a034-9259c41202d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:01.133588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dd91d0-d3c6-4a1f-a034-9259c41202d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:01.134093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd91d0-d3c6-4a1f-a034-9259c41202d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:01.135878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2dd91d0-d3c6-4a1f-a034-9259c41202d1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:01.167044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 867/1716] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:01.170099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6506f97-9002-41f6-b3ad-f6169925c172 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:01.196791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6506f97-9002-41f6-b3ad-f6169925c172 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:01.220190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fede09-fb56-448f-ade5-ae6311f007a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:01.222699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fede09-fb56-448f-ade5-ae6311f007a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:01.222699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fede09-fb56-448f-ade5-ae6311f007a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:01.223079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fede09-fb56-448f-ade5-ae6311f007a3 admin admin] PATH_INFO: `/v3/projects/65036afa371c40e8b7272dca0204de6a/users/389cfca2f47141b392cae5d61b067c24/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:01.278062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fede09-fb56-448f-ade5-ae6311f007a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65036afa371c40e8b7272dca0204de6a, user_id=389cfca2f47141b392cae5d61b067c24, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:01.286273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fede09-fb56-448f-ade5-ae6311f007a3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:01.316882 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 850/1717] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:01 2022] PUT /identity/v3/projects/65036afa371c40e8b7272dca0204de6a/users/389cfca2f47141b392cae5d61b067c24/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:01.321148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd91d0-d3c6-4a1f-a034-9259c41202d1 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:01.338618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dd91d0-d3c6-4a1f-a034-9259c41202d1 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:01.480784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67ac8-18bc-4336-9db2-1055e976b8dd None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:01.481332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67ac8-18bc-4336-9db2-1055e976b8dd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:01.481906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff67ac8-18bc-4336-9db2-1055e976b8dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:01.540542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451c165-8c48-44b3-a41e-637e17a121a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:01.550960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451c165-8c48-44b3-a41e-637e17a121a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:01.551697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451c165-8c48-44b3-a41e-637e17a121a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:01.552540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1451c165-8c48-44b3-a41e-637e17a121a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:01.553392 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451c165-8c48-44b3-a41e-637e17a121a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:01.556937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1451c165-8c48-44b3-a41e-637e17a121a4 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:01.561502 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5ff67ac8-18bc-4336-9db2-1055e976b8dd None None] MFA Rules not processed for user `e305e7d0cbfc4aeca6164685d0155475`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:01.574854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5ff67ac8-18bc-4336-9db2-1055e976b8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:01.612463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 868/1718] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:01 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:01.619679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451c165-8c48-44b3-a41e-637e17a121a4 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:01.647672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1451c165-8c48-44b3-a41e-637e17a121a4 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:01.664694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ee29ef-c957-4e7e-a15f-19a235dc98d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:01.665623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee29ef-c957-4e7e-a15f-19a235dc98d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:01.665913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee29ef-c957-4e7e-a15f-19a235dc98d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:01.666197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ee29ef-c957-4e7e-a15f-19a235dc98d9 admin admin] PATH_INFO: `/v3/projects/65036afa371c40e8b7272dca0204de6a/users/389cfca2f47141b392cae5d61b067c24/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:01.674034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee29ef-c957-4e7e-a15f-19a235dc98d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=65036afa371c40e8b7272dca0204de6a, user_id=389cfca2f47141b392cae5d61b067c24, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:01.685412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7ee29ef-c957-4e7e-a15f-19a235dc98d9 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:01.747662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 869/1719] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:01 2022] PUT /identity/v3/projects/65036afa371c40e8b7272dca0204de6a/users/389cfca2f47141b392cae5d61b067c24/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:01.753589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee401a-05ea-48cb-a9fe-cbfb699f0fd4 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:01.754055 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:01.758992 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee401a-05ea-48cb-a9fe-cbfb699f0fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:01.759961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee401a-05ea-48cb-a9fe-cbfb699f0fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:01.839842 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7aee401a-05ea-48cb-a9fe-cbfb699f0fd4 None None] MFA Rules not processed for user `389cfca2f47141b392cae5d61b067c24`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:01.842856 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7aee401a-05ea-48cb-a9fe-cbfb699f0fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:01.904542 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 851/1720] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:01 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:01.994221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 870/1721] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:01 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:02.005515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67ac8-18bc-4336-9db2-1055e976b8dd None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:02.007500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5ff67ac8-18bc-4336-9db2-1055e976b8dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:02.113935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff67ac8-18bc-4336-9db2-1055e976b8dd None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:02.319280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da96e8-34b2-40cd-8335-6cdb74f0f219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:02.320975 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da96e8-34b2-40cd-8335-6cdb74f0f219 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:02.323678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da96e8-34b2-40cd-8335-6cdb74f0f219 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:02.324180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da96e8-34b2-40cd-8335-6cdb74f0f219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:02.357078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da96e8-34b2-40cd-8335-6cdb74f0f219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:02.361581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da96e8-34b2-40cd-8335-6cdb74f0f219 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:02.397671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 852/1722] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:02 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:02.547668 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e3a67-9367-422a-9a99-23114d46a3bd None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:02.548816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e3a67-9367-422a-9a99-23114d46a3bd None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:02.550389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301e3a67-9367-422a-9a99-23114d46a3bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:02.705154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:02.711225 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-301e3a67-9367-422a-9a99-23114d46a3bd None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:02.718546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-301e3a67-9367-422a-9a99-23114d46a3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:02.983901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 871/1723] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:43:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 439 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:03.330204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da96e8-34b2-40cd-8335-6cdb74f0f219 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:03.350670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e3a67-9367-422a-9a99-23114d46a3bd None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:03.353125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-301e3a67-9367-422a-9a99-23114d46a3bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:03.577541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da96e8-34b2-40cd-8335-6cdb74f0f219 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:03.631680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:03.634234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:03.635786 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:03.638046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] PATH_INFO: `/v3/users/8d5ffaecc5674700b4b5f12ed3938122` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:03.643735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d5ffaecc5674700b4b5f12ed3938122)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:03.657018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:03.748479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301e3a67-9367-422a-9a99-23114d46a3bd None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:03.757839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6952bd8f-132a-4183-bc1a-f8c6b715ffb5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:03.763416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952bd8f-132a-4183-bc1a-f8c6b715ffb5 service swift] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:03.764105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952bd8f-132a-4183-bc1a-f8c6b715ffb5 service swift] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:03.764618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6952bd8f-132a-4183-bc1a-f8c6b715ffb5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:03.779021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:03.802168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952bd8f-132a-4183-bc1a-f8c6b715ffb5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:03.804640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6952bd8f-132a-4183-bc1a-f8c6b715ffb5 service swift] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:03.816513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d5ffaecc5674700b4b5f12ed3938122'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:03.817577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d5ffaecc5674700b4b5f12ed3938122'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:03.837731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 872/1724] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:43:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:43:03.857719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d5ffaecc5674700b4b5f12ed3938122'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:03.867641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d5ffaecc5674700b4b5f12ed3938122'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:03.995136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d5ffaecc5674700b4b5f12ed3938122'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:04.060584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 853/1725] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:03 2022] DELETE /identity/v3/users/8d5ffaecc5674700b4b5f12ed3938122 => generated 0 bytes in 728 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:04.070100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7bda9-7404-4703-9ded-70f4c197f3c9 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:04.071148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7bda9-7404-4703-9ded-70f4c197f3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:04.071680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc7bda9-7404-4703-9ded-70f4c197f3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:04.152089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-efc7bda9-7404-4703-9ded-70f4c197f3c9 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:04.154136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-efc7bda9-7404-4703-9ded-70f4c197f3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:04.410424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 873/1726] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:04.663764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:04.666184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:04.995898 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee16575-fafb-4653-b3e6-16605f130a57 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:05.033404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6f917-c7f0-497e-9ee5-2acee14fa6e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:05.034900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6f917-c7f0-497e-9ee5-2acee14fa6e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:05.038459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6f917-c7f0-497e-9ee5-2acee14fa6e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:05.040885 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b6f917-c7f0-497e-9ee5-2acee14fa6e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:05.095057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6f917-c7f0-497e-9ee5-2acee14fa6e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:05.096828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b6f917-c7f0-497e-9ee5-2acee14fa6e5 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:05.161764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 854/1727] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:05.316635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca71563-76c6-42bb-996a-c02c20113dcc None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:05.320155 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca71563-76c6-42bb-996a-c02c20113dcc None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:05.320540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca71563-76c6-42bb-996a-c02c20113dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:05.413963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7ca71563-76c6-42bb-996a-c02c20113dcc None None] MFA Rules not processed for user `2658435f74a947f39e207465ebff22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:05.419634 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7ca71563-76c6-42bb-996a-c02c20113dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:05.466559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 874/1728] 10.222.0.49 () {58 vars in 1015 bytes} [Tue Sep 6 22:43:05 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:05.645419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6f917-c7f0-497e-9ee5-2acee14fa6e5 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:05.655552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b6f917-c7f0-497e-9ee5-2acee14fa6e5 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:05.668681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae6cc9-32d2-4f0d-86fc-6acf29d9e99d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:05.670062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae6cc9-32d2-4f0d-86fc-6acf29d9e99d service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:05.670500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae6cc9-32d2-4f0d-86fc-6acf29d9e99d service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:05.670947 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ae6cc9-32d2-4f0d-86fc-6acf29d9e99d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:05.692449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae6cc9-32d2-4f0d-86fc-6acf29d9e99d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:05.694190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ae6cc9-32d2-4f0d-86fc-6acf29d9e99d service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:05.716475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 855/1729] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:07.587381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca71563-76c6-42bb-996a-c02c20113dcc None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:07.588314 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7ca71563-76c6-42bb-996a-c02c20113dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:07.617530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca71563-76c6-42bb-996a-c02c20113dcc None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:07.641222 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:07.644274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:07.645398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:07.648278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] PATH_INFO: `/v3/projects/4f72d0727933497c8e805854bd226db8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:07.652487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f72d0727933497c8e805854bd226db8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:07.662866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:07.664862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:07.826294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f72d0727933497c8e805854bd226db8', 'request_id': 'req-bb1dd254-e2fd-47b5-9c86-18f4a8705424'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:07.862500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f72d0727933497c8e805854bd226db8', 'request_id': 'req-bb1dd254-e2fd-47b5-9c86-18f4a8705424'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:07.890849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f72d0727933497c8e805854bd226db8', 'request_id': 'req-bb1dd254-e2fd-47b5-9c86-18f4a8705424'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:07.925167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] The token cache is being invalidate because project 4f72d0727933497c8e805854bd226db8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:07.925452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:07.926653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bb1dd254-e2fd-47b5-9c86-18f4a8705424 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:07.929063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 875/1730] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:07 2022] DELETE /identity/v3/projects/4f72d0727933497c8e805854bd226db8 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:07.933597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae6cc9-32d2-4f0d-86fc-6acf29d9e99d service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:07.965518 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ae6cc9-32d2-4f0d-86fc-6acf29d9e99d service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:08.068273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2e5db-ed61-4ef4-b195-b899947e02d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:08.069309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2e5db-ed61-4ef4-b195-b899947e02d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:08.070551 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2e5db-ed61-4ef4-b195-b899947e02d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:08.070986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2e5db-ed61-4ef4-b195-b899947e02d0 admin admin] PATH_INFO: `/v3/users/389cfca2f47141b392cae5d61b067c24` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:08.072605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2e5db-ed61-4ef4-b195-b899947e02d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=389cfca2f47141b392cae5d61b067c24)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:08.077712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2e5db-ed61-4ef4-b195-b899947e02d0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:08.203370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:08.214358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b7a2e5db-ed61-4ef4-b195-b899947e02d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '389cfca2f47141b392cae5d61b067c24'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:08.219710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b7a2e5db-ed61-4ef4-b195-b899947e02d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '389cfca2f47141b392cae5d61b067c24'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:08.237821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b7a2e5db-ed61-4ef4-b195-b899947e02d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '389cfca2f47141b392cae5d61b067c24'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:08.269341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b7a2e5db-ed61-4ef4-b195-b899947e02d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '389cfca2f47141b392cae5d61b067c24'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:08.306445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b7a2e5db-ed61-4ef4-b195-b899947e02d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '389cfca2f47141b392cae5d61b067c24'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:08.332176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 856/1731] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:07 2022] DELETE /identity/v3/users/389cfca2f47141b392cae5d61b067c24 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:09.897851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46611533-4768-45bc-8965-f535db7993df None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:09.898589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46611533-4768-45bc-8965-f535db7993df None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:09.898589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46611533-4768-45bc-8965-f535db7993df None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:09.900286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 876/1732] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:43:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:43:10.008178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca49f8-0cbd-4ec1-8706-1d5be64ff67c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:10.008972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca49f8-0cbd-4ec1-8706-1d5be64ff67c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:10.008972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cca49f8-0cbd-4ec1-8706-1d5be64ff67c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:10.106618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4cca49f8-0cbd-4ec1-8706-1d5be64ff67c None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:10.108276 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4cca49f8-0cbd-4ec1-8706-1d5be64ff67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:10.408158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 857/1733] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:43:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 401 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:11.715469 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46611533-4768-45bc-8965-f535db7993df None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:11.720164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-46611533-4768-45bc-8965-f535db7993df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:11.744246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46611533-4768-45bc-8965-f535db7993df None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:11.766234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff9edf8-7e67-42c1-a44e-4dcde1b8f4fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:11.766900 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9edf8-7e67-42c1-a44e-4dcde1b8f4fc service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:11.767178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9edf8-7e67-42c1-a44e-4dcde1b8f4fc service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:11.767457 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff9edf8-7e67-42c1-a44e-4dcde1b8f4fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:11.786343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9edf8-7e67-42c1-a44e-4dcde1b8f4fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:11.788183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff9edf8-7e67-42c1-a44e-4dcde1b8f4fc service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:11.802261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 877/1734] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:12.132506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca49f8-0cbd-4ec1-8706-1d5be64ff67c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:12.135320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4cca49f8-0cbd-4ec1-8706-1d5be64ff67c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:12.301272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cca49f8-0cbd-4ec1-8706-1d5be64ff67c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:12.309830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:12.311982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:12.325284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:12.325284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] PATH_INFO: `/v3/projects/65036afa371c40e8b7272dca0204de6a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:12.328949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65036afa371c40e8b7272dca0204de6a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:12.331420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:12.398582 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65036afa371c40e8b7272dca0204de6a', 'request_id': 'req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:12.400806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:12.405688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65036afa371c40e8b7272dca0204de6a', 'request_id': 'req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:12.416287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65036afa371c40e8b7272dca0204de6a', 'request_id': 'req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:12.428091 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] The token cache is being invalidate because project 65036afa371c40e8b7272dca0204de6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:12.428375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:12.429039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ae1d8d22-c5a3-4e6e-ad9f-fabd99e797b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:12.436240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 858/1735] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:12 2022] DELETE /identity/v3/projects/65036afa371c40e8b7272dca0204de6a => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:12.438159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df9ad9-e634-4625-a746-7d7a187d991d None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:12.438612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df9ad9-e634-4625-a746-7d7a187d991d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:12.439029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df9ad9-e634-4625-a746-7d7a187d991d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:12.461885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-61df9ad9-e634-4625-a746-7d7a187d991d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:12.463110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-61df9ad9-e634-4625-a746-7d7a187d991d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:12.541924 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 878/1736] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:12.554390 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9b280-afb8-4df6-9455-e9e09b138564 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:12.554673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9b280-afb8-4df6-9455-e9e09b138564 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:12.554950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c9b280-afb8-4df6-9455-e9e09b138564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:12.583865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-64c9b280-afb8-4df6-9455-e9e09b138564 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:12.585172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-64c9b280-afb8-4df6-9455-e9e09b138564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:12.593111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 859/1737] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:12.598150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df9ad9-e634-4625-a746-7d7a187d991d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:12.609853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df9ad9-e634-4625-a746-7d7a187d991d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:12.614532 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b6d1e-a32e-439e-a77c-18b1818c7725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:12.615136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b6d1e-a32e-439e-a77c-18b1818c7725 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:12.615407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b6d1e-a32e-439e-a77c-18b1818c7725 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:12.615673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4b6d1e-a32e-439e-a77c-18b1818c7725 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:12.616307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b6d1e-a32e-439e-a77c-18b1818c7725 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:12.616603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b6d1e-a32e-439e-a77c-18b1818c7725 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:12.618538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4b6d1e-a32e-439e-a77c-18b1818c7725 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:12.635621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 879/1738] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:43:12.640428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9b280-afb8-4df6-9455-e9e09b138564 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:12.655293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c9b280-afb8-4df6-9455-e9e09b138564 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:12.663736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b1e9e-7966-4d1f-9ef0-2258af8f0558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:12.664783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b1e9e-7966-4d1f-9ef0-2258af8f0558 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:12.666389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b1e9e-7966-4d1f-9ef0-2258af8f0558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:12.666733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60b1e9e-7966-4d1f-9ef0-2258af8f0558 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:12.667445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b1e9e-7966-4d1f-9ef0-2258af8f0558 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:12.669470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60b1e9e-7966-4d1f-9ef0-2258af8f0558 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:12.724883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 860/1739] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:12 2022] POST /identity/v3/projects => generated 365 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:12.727597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b6d1e-a32e-439e-a77c-18b1818c7725 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:12.736335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4b6d1e-a32e-439e-a77c-18b1818c7725 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:12.795480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3edb1-e6a1-4255-a9bf-a26aa38a10c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:12.796086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3edb1-e6a1-4255-a9bf-a26aa38a10c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:12.796359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3edb1-e6a1-4255-a9bf-a26aa38a10c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:12.796679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3edb1-e6a1-4255-a9bf-a26aa38a10c3 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:12.797378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3edb1-e6a1-4255-a9bf-a26aa38a10c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:12.799279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3edb1-e6a1-4255-a9bf-a26aa38a10c3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:12.809926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:12.825292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 880/1740] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:12 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:12.829997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b1e9e-7966-4d1f-9ef0-2258af8f0558 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:12.840273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60b1e9e-7966-4d1f-9ef0-2258af8f0558 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:12.845014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c1982-d068-4b50-9696-14f94f860513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:12.845692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c1982-d068-4b50-9696-14f94f860513 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:12.846046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c1982-d068-4b50-9696-14f94f860513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:12.846361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6c1982-d068-4b50-9696-14f94f860513 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:12.847099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c1982-d068-4b50-9696-14f94f860513 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:12.848789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6c1982-d068-4b50-9696-14f94f860513 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:12.858872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 861/1741] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:12.864250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3edb1-e6a1-4255-a9bf-a26aa38a10c3 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:12.877919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3edb1-e6a1-4255-a9bf-a26aa38a10c3 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:12.891158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e6675d-b15f-4033-b3f4-c70a5bfadd46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:12.892048 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6675d-b15f-4033-b3f4-c70a5bfadd46 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:12.892555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6675d-b15f-4033-b3f4-c70a5bfadd46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:12.893088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e6675d-b15f-4033-b3f4-c70a5bfadd46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:12.894730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6675d-b15f-4033-b3f4-c70a5bfadd46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:12.896649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e6675d-b15f-4033-b3f4-c70a5bfadd46 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:12.915327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 881/1742] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:12 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:12.918480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c1982-d068-4b50-9696-14f94f860513 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:12.930120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6c1982-d068-4b50-9696-14f94f860513 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:12.935284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920530c-4163-40c5-a3b7-45a05b3834c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:12.936331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920530c-4163-40c5-a3b7-45a05b3834c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:12.936331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920530c-4163-40c5-a3b7-45a05b3834c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:12.936532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e920530c-4163-40c5-a3b7-45a05b3834c0 admin admin] PATH_INFO: `/v3/projects/93778af5acd341feac357943a18da0c4/users/71fdab4ad7d748d097a36429330eefb5/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:12.955460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920530c-4163-40c5-a3b7-45a05b3834c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93778af5acd341feac357943a18da0c4, user_id=71fdab4ad7d748d097a36429330eefb5, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:12.958209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e920530c-4163-40c5-a3b7-45a05b3834c0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:12.973109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 862/1743] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:12 2022] PUT /identity/v3/projects/93778af5acd341feac357943a18da0c4/users/71fdab4ad7d748d097a36429330eefb5/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:12.985546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f0f8d-7308-489d-be32-e5aecd714133 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:12.985907 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f0f8d-7308-489d-be32-e5aecd714133 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:12.986254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f0f8d-7308-489d-be32-e5aecd714133 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.018749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-ac2f0f8d-7308-489d-be32-e5aecd714133 None None] MFA Rules not processed for user `71fdab4ad7d748d097a36429330eefb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:13.020292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ac2f0f8d-7308-489d-be32-e5aecd714133 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:13.105414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 882/1744] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:13.111241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920530c-4163-40c5-a3b7-45a05b3834c0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:13.185566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e920530c-4163-40c5-a3b7-45a05b3834c0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:13.191821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04876260-4708-4ce1-80cb-9b988713e01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:13.192504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04876260-4708-4ce1-80cb-9b988713e01f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:13.192881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04876260-4708-4ce1-80cb-9b988713e01f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:13.193227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04876260-4708-4ce1-80cb-9b988713e01f admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.193875 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04876260-4708-4ce1-80cb-9b988713e01f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:13.195858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04876260-4708-4ce1-80cb-9b988713e01f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:13.200001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6b13c-8619-4f98-9ca4-04c09ab220d4 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:13.200284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6b13c-8619-4f98-9ca4-04c09ab220d4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:13.200738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa6b13c-8619-4f98-9ca4-04c09ab220d4 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.201920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 883/1745] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:43:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:43:13.210095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ae144-3364-4c86-b10f-0d1bf6dcd8e0 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:13.210375 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ae144-3364-4c86-b10f-0d1bf6dcd8e0 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:13.210647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ae144-3364-4c86-b10f-0d1bf6dcd8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.236345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7a9ae144-3364-4c86-b10f-0d1bf6dcd8e0 None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:13.237625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7a9ae144-3364-4c86-b10f-0d1bf6dcd8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:13.240577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 863/1746] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:13 2022] POST /identity/v3/projects => generated 365 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:13.242275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04876260-4708-4ce1-80cb-9b988713e01f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:13.346002 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04876260-4708-4ce1-80cb-9b988713e01f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:13.352174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 884/1747] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:43:13 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:13.368382 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ae144-3364-4c86-b10f-0d1bf6dcd8e0 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:13.370096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d436f3-5b9e-48f8-81d6-b141731de63f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:13.370990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d436f3-5b9e-48f8-81d6-b141731de63f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:13.371374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7a9ae144-3364-4c86-b10f-0d1bf6dcd8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:13.371697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d436f3-5b9e-48f8-81d6-b141731de63f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:13.372176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d436f3-5b9e-48f8-81d6-b141731de63f admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.372964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d436f3-5b9e-48f8-81d6-b141731de63f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:13.376319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d436f3-5b9e-48f8-81d6-b141731de63f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:13.388842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:13.394037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ae144-3364-4c86-b10f-0d1bf6dcd8e0 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:13.405333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87ca0a-f7c7-4b1e-a50e-7103ba904ed5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:13.407355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ca0a-f7c7-4b1e-a50e-7103ba904ed5 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:13.407892 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ca0a-f7c7-4b1e-a50e-7103ba904ed5 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:13.408528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b87ca0a-f7c7-4b1e-a50e-7103ba904ed5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.412677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 864/1748] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:13 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:13.418144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d436f3-5b9e-48f8-81d6-b141731de63f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:13.420299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87ca0a-f7c7-4b1e-a50e-7103ba904ed5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:13.422863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b87ca0a-f7c7-4b1e-a50e-7103ba904ed5 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:13.433261 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d436f3-5b9e-48f8-81d6-b141731de63f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:13.435490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 885/1749] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:43:13.441524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef91824-ebea-4c73-8129-e4f5c7983673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:13.442832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef91824-ebea-4c73-8129-e4f5c7983673 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:13.443854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef91824-ebea-4c73-8129-e4f5c7983673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:13.444294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef91824-ebea-4c73-8129-e4f5c7983673 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.445059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef91824-ebea-4c73-8129-e4f5c7983673 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:13.446872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ef91824-ebea-4c73-8129-e4f5c7983673 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:13.458981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 865/1750] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:43:13.463412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87ca0a-f7c7-4b1e-a50e-7103ba904ed5 service nova] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:13.475967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b87ca0a-f7c7-4b1e-a50e-7103ba904ed5 service nova] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:13.481828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fef5f-92e2-48cd-9890-01277625588b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:13.482745 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fef5f-92e2-48cd-9890-01277625588b admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:13.483121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fef5f-92e2-48cd-9890-01277625588b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:13.483666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3fef5f-92e2-48cd-9890-01277625588b admin admin] PATH_INFO: `/v3/projects/229592fb38ca4599b8064599148d1162/users/12dd01aa2f2d471dba466771dcb3b37b/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.502675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fef5f-92e2-48cd-9890-01277625588b admin admin] RBAC: Authorizing `identity:create_grant(project_id=229592fb38ca4599b8064599148d1162, user_id=12dd01aa2f2d471dba466771dcb3b37b, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:13.505411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3fef5f-92e2-48cd-9890-01277625588b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:13.524664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 886/1751] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:13 2022] PUT /identity/v3/projects/229592fb38ca4599b8064599148d1162/users/12dd01aa2f2d471dba466771dcb3b37b/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:13.532953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef91824-ebea-4c73-8129-e4f5c7983673 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:13.544204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef91824-ebea-4c73-8129-e4f5c7983673 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:13.628242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5898e971-7b94-4817-8de0-4f7bcb765611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:13.629047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898e971-7b94-4817-8de0-4f7bcb765611 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:13.629323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898e971-7b94-4817-8de0-4f7bcb765611 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:13.629588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5898e971-7b94-4817-8de0-4f7bcb765611 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.630252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898e971-7b94-4817-8de0-4f7bcb765611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:13.631820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5898e971-7b94-4817-8de0-4f7bcb765611 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:13.642714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 866/1752] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:13 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:13.647397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fef5f-92e2-48cd-9890-01277625588b admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:13.658476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3fef5f-92e2-48cd-9890-01277625588b admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:13.663108 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a15322-b864-450d-bf71-24f3c28aa840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:13.663961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a15322-b864-450d-bf71-24f3c28aa840 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:13.664394 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a15322-b864-450d-bf71-24f3c28aa840 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:13.664784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a15322-b864-450d-bf71-24f3c28aa840 admin admin] PATH_INFO: `/v3/projects/229592fb38ca4599b8064599148d1162/users/12dd01aa2f2d471dba466771dcb3b37b/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.666908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a15322-b864-450d-bf71-24f3c28aa840 admin admin] RBAC: Authorizing `identity:create_grant(project_id=229592fb38ca4599b8064599148d1162, user_id=12dd01aa2f2d471dba466771dcb3b37b, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:13.669798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a15322-b864-450d-bf71-24f3c28aa840 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:13.680481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 887/1753] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:13 2022] PUT /identity/v3/projects/229592fb38ca4599b8064599148d1162/users/12dd01aa2f2d471dba466771dcb3b37b/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:13.691125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6c3e-47df-4c6f-80c0-3797fed6262b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:13.691417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6c3e-47df-4c6f-80c0-3797fed6262b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:13.691693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065e6c3e-47df-4c6f-80c0-3797fed6262b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.720458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-065e6c3e-47df-4c6f-80c0-3797fed6262b None None] MFA Rules not processed for user `12dd01aa2f2d471dba466771dcb3b37b`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:13.721734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-065e6c3e-47df-4c6f-80c0-3797fed6262b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:13.814678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 867/1754] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:13.828322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a15322-b864-450d-bf71-24f3c28aa840 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:13.830094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-70a15322-b864-450d-bf71-24f3c28aa840 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:13.851681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:13.902252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a15322-b864-450d-bf71-24f3c28aa840 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:13.906303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14544884-f127-43ec-81ac-261a9f46538c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:13.907063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14544884-f127-43ec-81ac-261a9f46538c service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:13.907512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14544884-f127-43ec-81ac-261a9f46538c service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:13.908085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14544884-f127-43ec-81ac-261a9f46538c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:13.919466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14544884-f127-43ec-81ac-261a9f46538c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:13.921167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14544884-f127-43ec-81ac-261a9f46538c service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:13.933183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 888/1755] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:13 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:14.016992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6c3e-47df-4c6f-80c0-3797fed6262b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:14.027227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065e6c3e-47df-4c6f-80c0-3797fed6262b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:14.031468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b727080-5ad4-4b9c-a546-c87a69b433cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:14.039669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b727080-5ad4-4b9c-a546-c87a69b433cc service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:14.039940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b727080-5ad4-4b9c-a546-c87a69b433cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:14.040205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b727080-5ad4-4b9c-a546-c87a69b433cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:14.121162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b727080-5ad4-4b9c-a546-c87a69b433cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:14.122837 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b727080-5ad4-4b9c-a546-c87a69b433cc service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:14.129571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:14.134950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 868/1756] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:16.668752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14544884-f127-43ec-81ac-261a9f46538c service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:16.673699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-14544884-f127-43ec-81ac-261a9f46538c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:16.777571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14544884-f127-43ec-81ac-261a9f46538c service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:16.808367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:16.810223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:16.810575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:16.811709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] PATH_INFO: `/v3/users/48d26e4ad5914e8f9fb97109002ef80f` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:16.813757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=48d26e4ad5914e8f9fb97109002ef80f)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:16.816831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:16.912263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48d26e4ad5914e8f9fb97109002ef80f'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:16.925377 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48d26e4ad5914e8f9fb97109002ef80f'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:16.926083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48d26e4ad5914e8f9fb97109002ef80f'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:16.943728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48d26e4ad5914e8f9fb97109002ef80f'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:16.971132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48d26e4ad5914e8f9fb97109002ef80f'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:16.990016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 889/1757] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:16 2022] DELETE /identity/v3/users/48d26e4ad5914e8f9fb97109002ef80f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:16.994537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590886f-cf1f-4ed6-a8f4-5f79b76f0905 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:16.995039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590886f-cf1f-4ed6-a8f4-5f79b76f0905 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:16.995134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590886f-cf1f-4ed6-a8f4-5f79b76f0905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:17.020641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8590886f-cf1f-4ed6-a8f4-5f79b76f0905 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:17.022211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8590886f-cf1f-4ed6-a8f4-5f79b76f0905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:17.134786 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 869/1758] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:17.145111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:17.146531 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:17.217943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:17.267194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e72f3e-46c6-42d5-906a-4fd33634ad0e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:17.274935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af589d-d378-4de3-a704-3e117314ce4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:17.275571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af589d-d378-4de3-a704-3e117314ce4a service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:17.275854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af589d-d378-4de3-a704-3e117314ce4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:17.276128 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af589d-d378-4de3-a704-3e117314ce4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:17.311943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af589d-d378-4de3-a704-3e117314ce4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:17.314140 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af589d-d378-4de3-a704-3e117314ce4a service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:17.339229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 890/1759] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:17.762163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590886f-cf1f-4ed6-a8f4-5f79b76f0905 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:17.764255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8590886f-cf1f-4ed6-a8f4-5f79b76f0905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:17.785777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590886f-cf1f-4ed6-a8f4-5f79b76f0905 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:17.790778 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:17.804143 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:17.804143 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:17.815795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] PATH_INFO: `/v3/projects/61ac98bf75ff4ee4ab6bc3ac6540e385` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:17.819668 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61ac98bf75ff4ee4ab6bc3ac6540e385)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:17.828409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:17.973727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61ac98bf75ff4ee4ab6bc3ac6540e385', 'request_id': 'req-d22b58d3-f9fd-4c69-91c0-24045589f938'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:17.982595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61ac98bf75ff4ee4ab6bc3ac6540e385', 'request_id': 'req-d22b58d3-f9fd-4c69-91c0-24045589f938'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.007686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61ac98bf75ff4ee4ab6bc3ac6540e385', 'request_id': 'req-d22b58d3-f9fd-4c69-91c0-24045589f938'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.052446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] The token cache is being invalidate because project 61ac98bf75ff4ee4ab6bc3ac6540e385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:18.052814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.055800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.060136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 870/1760] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:17 2022] DELETE /identity/v3/projects/61ac98bf75ff4ee4ab6bc3ac6540e385 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:18.064600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af589d-d378-4de3-a704-3e117314ce4a service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:18.077885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af589d-d378-4de3-a704-3e117314ce4a service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:18.270041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6854f0ea-bd18-461e-bc3f-1f9c84c802f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:18.270663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854f0ea-bd18-461e-bc3f-1f9c84c802f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:18.270949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854f0ea-bd18-461e-bc3f-1f9c84c802f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:18.271249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6854f0ea-bd18-461e-bc3f-1f9c84c802f6 admin admin] PATH_INFO: `/v3/users/aacee78ca3fb48af96b27e1f44880bdf` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:18.272452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854f0ea-bd18-461e-bc3f-1f9c84c802f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aacee78ca3fb48af96b27e1f44880bdf)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:18.274856 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6854f0ea-bd18-461e-bc3f-1f9c84c802f6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:18.423479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6854f0ea-bd18-461e-bc3f-1f9c84c802f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aacee78ca3fb48af96b27e1f44880bdf'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.433893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6854f0ea-bd18-461e-bc3f-1f9c84c802f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aacee78ca3fb48af96b27e1f44880bdf'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.434625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6854f0ea-bd18-461e-bc3f-1f9c84c802f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aacee78ca3fb48af96b27e1f44880bdf'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.451808 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6854f0ea-bd18-461e-bc3f-1f9c84c802f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aacee78ca3fb48af96b27e1f44880bdf'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.464576 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6854f0ea-bd18-461e-bc3f-1f9c84c802f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aacee78ca3fb48af96b27e1f44880bdf'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.473190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 891/1761] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:18 2022] DELETE /identity/v3/users/aacee78ca3fb48af96b27e1f44880bdf => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:18.797851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:18.799318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:18.812952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22b58d3-f9fd-4c69-91c0-24045589f938 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:18.870980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:18.871612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:18.871885 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:18.872630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] PATH_INFO: `/v3/projects/176f0d6a7c264d93a85139232dad3e8e` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:18.873803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] RBAC: Authorizing `identity:delete_project(project_id=176f0d6a7c264d93a85139232dad3e8e)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:18.876679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:18.906908 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:18.925199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '176f0d6a7c264d93a85139232dad3e8e', 'request_id': 'req-789dedb4-53ad-47b1-8248-77478ea1d582'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.931221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '176f0d6a7c264d93a85139232dad3e8e', 'request_id': 'req-789dedb4-53ad-47b1-8248-77478ea1d582'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.938894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '176f0d6a7c264d93a85139232dad3e8e', 'request_id': 'req-789dedb4-53ad-47b1-8248-77478ea1d582'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.948976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] The token cache is being invalidate because project 176f0d6a7c264d93a85139232dad3e8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:18.949262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.949895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-789dedb4-53ad-47b1-8248-77478ea1d582 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:18.951410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 871/1762] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:18 2022] DELETE /identity/v3/projects/176f0d6a7c264d93a85139232dad3e8e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:18.956272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b067212-bac3-4d32-b3ce-81440a2138ef None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:18.956556 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b067212-bac3-4d32-b3ce-81440a2138ef None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:18.956894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b067212-bac3-4d32-b3ce-81440a2138ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:18.976557 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-4b067212-bac3-4d32-b3ce-81440a2138ef None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:18.977800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4b067212-bac3-4d32-b3ce-81440a2138ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:19.027954 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:19.038630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 892/1763] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:19.046395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f7fde-f63c-4a96-bc61-f570442a460a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.047406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f7fde-f63c-4a96-bc61-f570442a460a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.047406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82f7fde-f63c-4a96-bc61-f570442a460a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.067411 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c82f7fde-f63c-4a96-bc61-f570442a460a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:19.068686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c82f7fde-f63c-4a96-bc61-f570442a460a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:19.076740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 872/1764] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:19.080930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b067212-bac3-4d32-b3ce-81440a2138ef None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.090502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b067212-bac3-4d32-b3ce-81440a2138ef None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.095228 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18141b-be56-490e-a306-ddb9f5b3983c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.095820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18141b-be56-490e-a306-ddb9f5b3983c admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.096100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18141b-be56-490e-a306-ddb9f5b3983c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.096373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c18141b-be56-490e-a306-ddb9f5b3983c admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.097079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18141b-be56-490e-a306-ddb9f5b3983c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:19.097380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18141b-be56-490e-a306-ddb9f5b3983c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.098957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c18141b-be56-490e-a306-ddb9f5b3983c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.110893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 893/1765] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:43:19.115069 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f7fde-f63c-4a96-bc61-f570442a460a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.124219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82f7fde-f63c-4a96-bc61-f570442a460a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.129758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a46e9c-bd6d-4bd2-bc2e-b86c923bee53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.130356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a46e9c-bd6d-4bd2-bc2e-b86c923bee53 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.130631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a46e9c-bd6d-4bd2-bc2e-b86c923bee53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.130901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a46e9c-bd6d-4bd2-bc2e-b86c923bee53 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.131520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a46e9c-bd6d-4bd2-bc2e-b86c923bee53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.133462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a46e9c-bd6d-4bd2-bc2e-b86c923bee53 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.160510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 873/1766] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:19 2022] POST /identity/v3/projects => generated 399 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:19.164606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18141b-be56-490e-a306-ddb9f5b3983c admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.174931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c18141b-be56-490e-a306-ddb9f5b3983c admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.229934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9819b2-d7a7-4c53-a5ec-d0d59a0914f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.230917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9819b2-d7a7-4c53-a5ec-d0d59a0914f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.231461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9819b2-d7a7-4c53-a5ec-d0d59a0914f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.231743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9819b2-d7a7-4c53-a5ec-d0d59a0914f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.232382 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9819b2-d7a7-4c53-a5ec-d0d59a0914f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.234444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9819b2-d7a7-4c53-a5ec-d0d59a0914f6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.259748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 894/1767] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:19 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:19.263779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a46e9c-bd6d-4bd2-bc2e-b86c923bee53 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.274956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a46e9c-bd6d-4bd2-bc2e-b86c923bee53 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.279569 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893033e-75e9-4168-ba11-970457c9ddf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.280167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893033e-75e9-4168-ba11-970457c9ddf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.280439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893033e-75e9-4168-ba11-970457c9ddf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.280704 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7893033e-75e9-4168-ba11-970457c9ddf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.281374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893033e-75e9-4168-ba11-970457c9ddf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.282954 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7893033e-75e9-4168-ba11-970457c9ddf5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.293209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 874/1768] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:19.297982 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9819b2-d7a7-4c53-a5ec-d0d59a0914f6 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.309790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9819b2-d7a7-4c53-a5ec-d0d59a0914f6 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.315179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07ef086-159c-4993-a86a-28e9734f1a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.315787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ef086-159c-4993-a86a-28e9734f1a6a admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.316171 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ef086-159c-4993-a86a-28e9734f1a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.316789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07ef086-159c-4993-a86a-28e9734f1a6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.317458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ef086-159c-4993-a86a-28e9734f1a6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.319042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07ef086-159c-4993-a86a-28e9734f1a6a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.333909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 895/1769] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:19.335336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893033e-75e9-4168-ba11-970457c9ddf5 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.345956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7893033e-75e9-4168-ba11-970457c9ddf5 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.354186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76927e9-77e6-4181-8dce-02cd610f30d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.354968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76927e9-77e6-4181-8dce-02cd610f30d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.355270 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76927e9-77e6-4181-8dce-02cd610f30d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.355540 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76927e9-77e6-4181-8dce-02cd610f30d3 admin admin] PATH_INFO: `/v3/projects/c26a156485b64e4fa9fa10e4a19782d4/users/1557f13b042644cbb3c045dcac7e80f7/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.374272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76927e9-77e6-4181-8dce-02cd610f30d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c26a156485b64e4fa9fa10e4a19782d4, user_id=1557f13b042644cbb3c045dcac7e80f7, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.377123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76927e9-77e6-4181-8dce-02cd610f30d3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.392422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 875/1770] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:19 2022] PUT /identity/v3/projects/c26a156485b64e4fa9fa10e4a19782d4/users/1557f13b042644cbb3c045dcac7e80f7/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:19.403095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d9f4c-2f2f-46f2-ab75-61c00baca656 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.403374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d9f4c-2f2f-46f2-ab75-61c00baca656 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.403652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6d9f4c-2f2f-46f2-ab75-61c00baca656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.441650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-3d6d9f4c-2f2f-46f2-ab75-61c00baca656 None None] MFA Rules not processed for user `1557f13b042644cbb3c045dcac7e80f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:19.443823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3d6d9f4c-2f2f-46f2-ab75-61c00baca656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:19.532620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 896/1771] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:19 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:19.537341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76927e9-77e6-4181-8dce-02cd610f30d3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.619229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76927e9-77e6-4181-8dce-02cd610f30d3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.623889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c2b2f-4ac9-415b-b478-df9bc13f5647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.624492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c2b2f-4ac9-415b-b478-df9bc13f5647 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.625655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c2b2f-4ac9-415b-b478-df9bc13f5647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.625932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c2b2f-4ac9-415b-b478-df9bc13f5647 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.626563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c2b2f-4ac9-415b-b478-df9bc13f5647 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.628448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5c2b2f-4ac9-415b-b478-df9bc13f5647 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.656594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 876/1772] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:19 2022] POST /identity/v3/projects => generated 401 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:19.660878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d9f4c-2f2f-46f2-ab75-61c00baca656 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.672175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6d9f4c-2f2f-46f2-ab75-61c00baca656 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.725521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:19.736425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf4b6e-79b1-4fbd-9d59-94a33c42e942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.737065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf4b6e-79b1-4fbd-9d59-94a33c42e942 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.737364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf4b6e-79b1-4fbd-9d59-94a33c42e942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.737632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bf4b6e-79b1-4fbd-9d59-94a33c42e942 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.738254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf4b6e-79b1-4fbd-9d59-94a33c42e942 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.740300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bf4b6e-79b1-4fbd-9d59-94a33c42e942 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.763861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 897/1773] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:19 2022] POST /identity/v3/users => generated 307 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:19.769800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c2b2f-4ac9-415b-b478-df9bc13f5647 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.779979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5c2b2f-4ac9-415b-b478-df9bc13f5647 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.785214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde3d65-6ddc-4a0c-9253-a3a9b4628782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.785808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde3d65-6ddc-4a0c-9253-a3a9b4628782 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.786084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde3d65-6ddc-4a0c-9253-a3a9b4628782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.786349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dde3d65-6ddc-4a0c-9253-a3a9b4628782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.786982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde3d65-6ddc-4a0c-9253-a3a9b4628782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.788538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dde3d65-6ddc-4a0c-9253-a3a9b4628782 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.799715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 877/1774] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:19.803787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf4b6e-79b1-4fbd-9d59-94a33c42e942 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.814377 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bf4b6e-79b1-4fbd-9d59-94a33c42e942 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.818899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a9f2e-d3f7-4a74-a7b5-6f5de192636b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.819561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a9f2e-d3f7-4a74-a7b5-6f5de192636b admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.819820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a9f2e-d3f7-4a74-a7b5-6f5de192636b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.820094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88a9f2e-d3f7-4a74-a7b5-6f5de192636b admin admin] PATH_INFO: `/v3/projects/04db4237c4f64f9fb4e26b9f44026e5f/users/8084f2796bd84fe19d20db5855aba3ea/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.834603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a9f2e-d3f7-4a74-a7b5-6f5de192636b admin admin] RBAC: Authorizing `identity:create_grant(project_id=04db4237c4f64f9fb4e26b9f44026e5f, user_id=8084f2796bd84fe19d20db5855aba3ea, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.838014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88a9f2e-d3f7-4a74-a7b5-6f5de192636b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.850723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 898/1775] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:19 2022] PUT /identity/v3/projects/04db4237c4f64f9fb4e26b9f44026e5f/users/8084f2796bd84fe19d20db5855aba3ea/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:19.854708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde3d65-6ddc-4a0c-9253-a3a9b4628782 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.865141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dde3d65-6ddc-4a0c-9253-a3a9b4628782 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.885095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:19.923106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8793d0-5430-40a1-95f7-635dd4fe5ebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.923718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8793d0-5430-40a1-95f7-635dd4fe5ebc admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.923991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8793d0-5430-40a1-95f7-635dd4fe5ebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.924259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8793d0-5430-40a1-95f7-635dd4fe5ebc admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.924934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8793d0-5430-40a1-95f7-635dd4fe5ebc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.926512 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8793d0-5430-40a1-95f7-635dd4fe5ebc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.936128 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 878/1776] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:19 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:19.941080 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a9f2e-d3f7-4a74-a7b5-6f5de192636b admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:19.951231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88a9f2e-d3f7-4a74-a7b5-6f5de192636b admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:19.955829 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25642d-9cfb-481c-8dff-bcd57a031f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:19.956472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25642d-9cfb-481c-8dff-bcd57a031f1a admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.956751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25642d-9cfb-481c-8dff-bcd57a031f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.957052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d25642d-9cfb-481c-8dff-bcd57a031f1a admin admin] PATH_INFO: `/v3/projects/04db4237c4f64f9fb4e26b9f44026e5f/users/8084f2796bd84fe19d20db5855aba3ea/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:19.958942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25642d-9cfb-481c-8dff-bcd57a031f1a admin admin] RBAC: Authorizing `identity:create_grant(project_id=04db4237c4f64f9fb4e26b9f44026e5f, user_id=8084f2796bd84fe19d20db5855aba3ea, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:19.961605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d25642d-9cfb-481c-8dff-bcd57a031f1a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:19.972188 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 899/1777] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:19 2022] PUT /identity/v3/projects/04db4237c4f64f9fb4e26b9f44026e5f/users/8084f2796bd84fe19d20db5855aba3ea/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:19.983133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5b14c-9298-47bf-bfbb-d1ed0cec1d1e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:19.983421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5b14c-9298-47bf-bfbb-d1ed0cec1d1e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:19.983697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d5b14c-9298-47bf-bfbb-d1ed0cec1d1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:20.011377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-73d5b14c-9298-47bf-bfbb-d1ed0cec1d1e None None] MFA Rules not processed for user `8084f2796bd84fe19d20db5855aba3ea`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:20.012584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-73d5b14c-9298-47bf-bfbb-d1ed0cec1d1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:20.079498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 879/1778] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:19 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:20.090302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25642d-9cfb-481c-8dff-bcd57a031f1a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:20.092066 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5d25642d-9cfb-481c-8dff-bcd57a031f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:20.150636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d25642d-9cfb-481c-8dff-bcd57a031f1a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:20.154198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef329ea5-cf55-4494-a1ed-facda627ada2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:20.154840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef329ea5-cf55-4494-a1ed-facda627ada2 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:20.155139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef329ea5-cf55-4494-a1ed-facda627ada2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:20.155406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef329ea5-cf55-4494-a1ed-facda627ada2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:20.165057 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef329ea5-cf55-4494-a1ed-facda627ada2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:20.166458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef329ea5-cf55-4494-a1ed-facda627ada2 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:20.176477 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 900/1779] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:20 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:20.253148 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5b14c-9298-47bf-bfbb-d1ed0cec1d1e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:20.262584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d5b14c-9298-47bf-bfbb-d1ed0cec1d1e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:20.266524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11cad93-a7a5-484a-a5ff-d8573757b2cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:20.267241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cad93-a7a5-484a-a5ff-d8573757b2cc service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:20.267523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cad93-a7a5-484a-a5ff-d8573757b2cc service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:20.267827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11cad93-a7a5-484a-a5ff-d8573757b2cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:20.325031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11cad93-a7a5-484a-a5ff-d8573757b2cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:20.326442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11cad93-a7a5-484a-a5ff-d8573757b2cc service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:20.335421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 880/1780] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:20 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:28.009165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef329ea5-cf55-4494-a1ed-facda627ada2 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:28.011646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ef329ea5-cf55-4494-a1ed-facda627ada2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:28.205748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef329ea5-cf55-4494-a1ed-facda627ada2 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:28.223545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:28.227230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:28.229359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:28.231345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] PATH_INFO: `/v3/users/bd409aef013448cc86db813ad9769fa7` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:28.233135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd409aef013448cc86db813ad9769fa7)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:28.237766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:28.323122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd409aef013448cc86db813ad9769fa7'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:28.325770 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:28.341962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd409aef013448cc86db813ad9769fa7'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:28.342976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd409aef013448cc86db813ad9769fa7'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:28.372286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd409aef013448cc86db813ad9769fa7'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:28.381090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd409aef013448cc86db813ad9769fa7'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:28.387823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 901/1781] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:28 2022] DELETE /identity/v3/users/bd409aef013448cc86db813ad9769fa7 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:28.392732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee0cfc-ee0d-4de4-b251-e221cfa55c86 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:28.394382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee0cfc-ee0d-4de4-b251-e221cfa55c86 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:28.394889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbee0cfc-ee0d-4de4-b251-e221cfa55c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:28.418597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-fbee0cfc-ee0d-4de4-b251-e221cfa55c86 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:28.420499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fbee0cfc-ee0d-4de4-b251-e221cfa55c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:28.493939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:28.564832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 881/1782] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:28.594978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:28.596875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:28.768089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2dec4e-2d92-4c08-a8ee-f43b5e993ca0 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:28.773118 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fd16f-d298-477a-aaa8-729df27cf072 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:28.774445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fd16f-d298-477a-aaa8-729df27cf072 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:28.774965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fd16f-d298-477a-aaa8-729df27cf072 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:28.775513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fd16f-d298-477a-aaa8-729df27cf072 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:28.788793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fd16f-d298-477a-aaa8-729df27cf072 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:28.791622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4fd16f-d298-477a-aaa8-729df27cf072 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:28.813403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 902/1783] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:30.017322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee0cfc-ee0d-4de4-b251-e221cfa55c86 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:30.019703 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fbee0cfc-ee0d-4de4-b251-e221cfa55c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:30.034960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbee0cfc-ee0d-4de4-b251-e221cfa55c86 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:30.041860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:30.043144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:30.043734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:30.044304 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] PATH_INFO: `/v3/projects/19cc2b5d06dc46d38905348059b73e9e` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:30.046177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19cc2b5d06dc46d38905348059b73e9e)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:30.049716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:30.131916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19cc2b5d06dc46d38905348059b73e9e', 'request_id': 'req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:30.151093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19cc2b5d06dc46d38905348059b73e9e', 'request_id': 'req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:30.160951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19cc2b5d06dc46d38905348059b73e9e', 'request_id': 'req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:30.173167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] The token cache is being invalidate because project 19cc2b5d06dc46d38905348059b73e9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:30.173573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:30.174329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:30.175859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 882/1784] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:30 2022] DELETE /identity/v3/projects/19cc2b5d06dc46d38905348059b73e9e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:30.180048 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fd16f-d298-477a-aaa8-729df27cf072 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:30.190698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fd16f-d298-477a-aaa8-729df27cf072 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:30.356310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc55028-ab67-4e1a-a10e-29ea3c85c4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:30.357138 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc55028-ab67-4e1a-a10e-29ea3c85c4b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:30.357528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc55028-ab67-4e1a-a10e-29ea3c85c4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:30.357899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc55028-ab67-4e1a-a10e-29ea3c85c4b4 admin admin] PATH_INFO: `/v3/users/38317a8501cc414cadb852610a8f69f2` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:30.360425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc55028-ab67-4e1a-a10e-29ea3c85c4b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=38317a8501cc414cadb852610a8f69f2)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:30.363688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc55028-ab67-4e1a-a10e-29ea3c85c4b4 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:30.470897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5fc55028-ab67-4e1a-a10e-29ea3c85c4b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38317a8501cc414cadb852610a8f69f2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:30.485262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5fc55028-ab67-4e1a-a10e-29ea3c85c4b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38317a8501cc414cadb852610a8f69f2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:30.486274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5fc55028-ab67-4e1a-a10e-29ea3c85c4b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38317a8501cc414cadb852610a8f69f2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:30.502807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5fc55028-ab67-4e1a-a10e-29ea3c85c4b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38317a8501cc414cadb852610a8f69f2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:30.504993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:30.523983 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5fc55028-ab67-4e1a-a10e-29ea3c85c4b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38317a8501cc414cadb852610a8f69f2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:30.539308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 903/1785] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:30 2022] DELETE /identity/v3/users/38317a8501cc414cadb852610a8f69f2 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:31.024782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:31.027288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:31.042688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618bd4cd-1e0d-4cbd-a805-e3d7f5f73248 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:31.135487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:31.136142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:31.136418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:31.136748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] PATH_INFO: `/v3/projects/63e51fefc9ed4174826bff53fe6da705` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:31.137939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=63e51fefc9ed4174826bff53fe6da705)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:31.140375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:31.223503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63e51fefc9ed4174826bff53fe6da705', 'request_id': 'req-45daa256-ec70-474e-90bc-73573074f0e5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.232433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63e51fefc9ed4174826bff53fe6da705', 'request_id': 'req-45daa256-ec70-474e-90bc-73573074f0e5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.244696 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63e51fefc9ed4174826bff53fe6da705', 'request_id': 'req-45daa256-ec70-474e-90bc-73573074f0e5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.264282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:31.276174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] The token cache is being invalidate because project 63e51fefc9ed4174826bff53fe6da705 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:31.276174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.276174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.283690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 883/1786] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:31 2022] DELETE /identity/v3/projects/63e51fefc9ed4174826bff53fe6da705 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:31.283776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c7af7-4a89-4f67-86cb-f65a1bda374a None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:31.283776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c7af7-4a89-4f67-86cb-f65a1bda374a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:31.284254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c7af7-4a89-4f67-86cb-f65a1bda374a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:31.324087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d12c7af7-4a89-4f67-86cb-f65a1bda374a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:31.325730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d12c7af7-4a89-4f67-86cb-f65a1bda374a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:31.339130 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:31.340813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:31.429036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45daa256-ec70-474e-90bc-73573074f0e5 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:31.434498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0450efe-2cad-4408-bed4-bb12c44ad80b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:31.435138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0450efe-2cad-4408-bed4-bb12c44ad80b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:31.435415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0450efe-2cad-4408-bed4-bb12c44ad80b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:31.435680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0450efe-2cad-4408-bed4-bb12c44ad80b admin admin] PATH_INFO: `/v3/users/1557f13b042644cbb3c045dcac7e80f7` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:31.436885 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0450efe-2cad-4408-bed4-bb12c44ad80b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1557f13b042644cbb3c045dcac7e80f7)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:31.439590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0450efe-2cad-4408-bed4-bb12c44ad80b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:31.444791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 904/1787] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:31.447217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c7af7-4a89-4f67-86cb-f65a1bda374a None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:31.453336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d12c7af7-4a89-4f67-86cb-f65a1bda374a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:31.470113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c7af7-4a89-4f67-86cb-f65a1bda374a None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:31.475576 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d55ce9-bb30-420c-8153-8a35f2188e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:31.476221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d55ce9-bb30-420c-8153-8a35f2188e37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:31.476499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d55ce9-bb30-420c-8153-8a35f2188e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:31.476790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d55ce9-bb30-420c-8153-8a35f2188e37 admin admin] PATH_INFO: `/v3/users/71fdab4ad7d748d097a36429330eefb5` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:31.478482 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d55ce9-bb30-420c-8153-8a35f2188e37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=71fdab4ad7d748d097a36429330eefb5)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:31.480188 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d55ce9-bb30-420c-8153-8a35f2188e37 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:31.502961 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e0450efe-2cad-4408-bed4-bb12c44ad80b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1557f13b042644cbb3c045dcac7e80f7'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.505133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e0450efe-2cad-4408-bed4-bb12c44ad80b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1557f13b042644cbb3c045dcac7e80f7'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.515341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e0450efe-2cad-4408-bed4-bb12c44ad80b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1557f13b042644cbb3c045dcac7e80f7'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.525769 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e0450efe-2cad-4408-bed4-bb12c44ad80b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1557f13b042644cbb3c045dcac7e80f7'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.535293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-f1d55ce9-bb30-420c-8153-8a35f2188e37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71fdab4ad7d748d097a36429330eefb5'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.541026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e0450efe-2cad-4408-bed4-bb12c44ad80b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1557f13b042644cbb3c045dcac7e80f7'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.544785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-f1d55ce9-bb30-420c-8153-8a35f2188e37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71fdab4ad7d748d097a36429330eefb5'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.545623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-f1d55ce9-bb30-420c-8153-8a35f2188e37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71fdab4ad7d748d097a36429330eefb5'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.548188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 884/1788] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:31 2022] DELETE /identity/v3/users/1557f13b042644cbb3c045dcac7e80f7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:31.549603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7e5bb-8a96-4dfd-978e-da9c0c165110 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:31.549959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7e5bb-8a96-4dfd-978e-da9c0c165110 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:31.550305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be7e5bb-8a96-4dfd-978e-da9c0c165110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:31.562356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-f1d55ce9-bb30-420c-8153-8a35f2188e37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71fdab4ad7d748d097a36429330eefb5'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.570297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-f1d55ce9-bb30-420c-8153-8a35f2188e37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71fdab4ad7d748d097a36429330eefb5'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:31.572949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8be7e5bb-8a96-4dfd-978e-da9c0c165110 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:31.573075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8be7e5bb-8a96-4dfd-978e-da9c0c165110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:31.582353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 905/1789] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:31 2022] DELETE /identity/v3/users/71fdab4ad7d748d097a36429330eefb5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:31.584021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c792e9-26bc-488d-98c1-403f05210d8e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:31.584146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c792e9-26bc-488d-98c1-403f05210d8e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:31.584404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c792e9-26bc-488d-98c1-403f05210d8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:31.605847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-43c792e9-26bc-488d-98c1-403f05210d8e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:31.609181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-43c792e9-26bc-488d-98c1-403f05210d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:31.651389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 885/1790] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:31.652949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc317-b02a-432f-9bd0-70b27456e9f0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:31.653234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc317-b02a-432f-9bd0-70b27456e9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:31.653570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fc317-b02a-432f-9bd0-70b27456e9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:31.675522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-661fc317-b02a-432f-9bd0-70b27456e9f0 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:31.676746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-661fc317-b02a-432f-9bd0-70b27456e9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:31.691450 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 906/1791] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:31.692266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c792e9-26bc-488d-98c1-403f05210d8e None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:31.692532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:31.699020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:31.702802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c792e9-26bc-488d-98c1-403f05210d8e None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:31.711080 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828ab34-f73c-4859-b40d-37e191b70ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:31.713652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828ab34-f73c-4859-b40d-37e191b70ece admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:31.713961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828ab34-f73c-4859-b40d-37e191b70ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:31.714637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828ab34-f73c-4859-b40d-37e191b70ece admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:31.715939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828ab34-f73c-4859-b40d-37e191b70ece admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:31.716340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828ab34-f73c-4859-b40d-37e191b70ece admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:31.720370 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828ab34-f73c-4859-b40d-37e191b70ece admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:31.739388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 907/1792] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:43:31.739388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828ab34-f73c-4859-b40d-37e191b70ece admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:31.739388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-0828ab34-f73c-4859-b40d-37e191b70ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:31.772124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 886/1793] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:31.773731 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fc317-b02a-432f-9bd0-70b27456e9f0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:31.794326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fc317-b02a-432f-9bd0-70b27456e9f0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:31.803366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffcbe2-72e3-4a31-8107-55256bb0aa77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:31.804076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffcbe2-72e3-4a31-8107-55256bb0aa77 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:31.804432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffcbe2-72e3-4a31-8107-55256bb0aa77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:31.804810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ffcbe2-72e3-4a31-8107-55256bb0aa77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:31.805530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffcbe2-72e3-4a31-8107-55256bb0aa77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:31.809611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ffcbe2-72e3-4a31-8107-55256bb0aa77 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:31.830200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828ab34-f73c-4859-b40d-37e191b70ece admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:31.840612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373a62b-f640-4bb1-91b4-32f0e93a21ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:31.842579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373a62b-f640-4bb1-91b4-32f0e93a21ec service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:31.842579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373a62b-f640-4bb1-91b4-32f0e93a21ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:31.842579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a373a62b-f640-4bb1-91b4-32f0e93a21ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:31.854036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373a62b-f640-4bb1-91b4-32f0e93a21ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:31.855498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a373a62b-f640-4bb1-91b4-32f0e93a21ec service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:31.868208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 908/1794] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:31.874303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373a62b-f640-4bb1-91b4-32f0e93a21ec service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:31.885967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 887/1795] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:31 2022] POST /identity/v3/projects => generated 373 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:31.886888 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffcbe2-72e3-4a31-8107-55256bb0aa77 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:31.901135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a373a62b-f640-4bb1-91b4-32f0e93a21ec service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:32.049256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ffcbe2-72e3-4a31-8107-55256bb0aa77 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:32.059010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24b66b-d840-460d-bd06-139b0d62c50a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:32.059926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24b66b-d840-460d-bd06-139b0d62c50a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:32.060434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24b66b-d840-460d-bd06-139b0d62c50a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:32.060957 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c24b66b-d840-460d-bd06-139b0d62c50a admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:32.061843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24b66b-d840-460d-bd06-139b0d62c50a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:32.067873 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c24b66b-d840-460d-bd06-139b0d62c50a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:32.069510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b0fe7-5568-4605-a160-b335e47fe34b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:32.070146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b0fe7-5568-4605-a160-b335e47fe34b service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:32.070416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b0fe7-5568-4605-a160-b335e47fe34b service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:32.070691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6b0fe7-5568-4605-a160-b335e47fe34b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:32.096910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b0fe7-5568-4605-a160-b335e47fe34b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:32.101293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6b0fe7-5568-4605-a160-b335e47fe34b service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:32.118932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 888/1796] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:31 2022] POST /identity/v3/users => generated 287 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:32.124342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24b66b-d840-460d-bd06-139b0d62c50a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:32.136375 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 909/1797] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:32.143330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c24b66b-d840-460d-bd06-139b0d62c50a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:32.151816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aefb9f-766c-41a7-aac4-6f88cc969ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:32.152929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aefb9f-766c-41a7-aac4-6f88cc969ec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:32.153443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aefb9f-766c-41a7-aac4-6f88cc969ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:32.154775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8aefb9f-766c-41a7-aac4-6f88cc969ec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:32.157574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aefb9f-766c-41a7-aac4-6f88cc969ec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:32.161519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8aefb9f-766c-41a7-aac4-6f88cc969ec6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:32.202194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 889/1798] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:32.206528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b0fe7-5568-4605-a160-b335e47fe34b service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:32.222462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6b0fe7-5568-4605-a160-b335e47fe34b service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:32.241778 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17f5f2d-d7be-4e93-9a95-33936d5723f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:32.242843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f5f2d-d7be-4e93-9a95-33936d5723f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:32.243762 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f5f2d-d7be-4e93-9a95-33936d5723f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:32.243762 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17f5f2d-d7be-4e93-9a95-33936d5723f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:32.244612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f5f2d-d7be-4e93-9a95-33936d5723f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:32.247720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17f5f2d-d7be-4e93-9a95-33936d5723f6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:32.267222 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 910/1799] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:32 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:32.273231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aefb9f-766c-41a7-aac4-6f88cc969ec6 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:32.291783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8aefb9f-766c-41a7-aac4-6f88cc969ec6 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:32.301629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b39556-c207-438d-9014-dbff9aa2c0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:32.303085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b39556-c207-438d-9014-dbff9aa2c0f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:32.303448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b39556-c207-438d-9014-dbff9aa2c0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:32.305076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b39556-c207-438d-9014-dbff9aa2c0f2 admin admin] PATH_INFO: `/v3/projects/98f6a915c6a8400e9effc414b3e526d2/users/146ab2a5cc104752803e20f04a7a96ba/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:32.356996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b39556-c207-438d-9014-dbff9aa2c0f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=98f6a915c6a8400e9effc414b3e526d2, user_id=146ab2a5cc104752803e20f04a7a96ba, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:32.390126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b39556-c207-438d-9014-dbff9aa2c0f2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:32.416623 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 890/1800] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:32 2022] PUT /identity/v3/projects/98f6a915c6a8400e9effc414b3e526d2/users/146ab2a5cc104752803e20f04a7a96ba/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:32.428234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218261e5-cab9-481e-8c4b-54fb48786e3c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:32.428798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218261e5-cab9-481e-8c4b-54fb48786e3c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:32.428798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218261e5-cab9-481e-8c4b-54fb48786e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:32.491440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-218261e5-cab9-481e-8c4b-54fb48786e3c None None] MFA Rules not processed for user `146ab2a5cc104752803e20f04a7a96ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:32.492788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-218261e5-cab9-481e-8c4b-54fb48786e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:32.582567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b39556-c207-438d-9014-dbff9aa2c0f2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:32.584431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b7b39556-c207-438d-9014-dbff9aa2c0f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:32.611819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 911/1801] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:32.624356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218261e5-cab9-481e-8c4b-54fb48786e3c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:32.626964 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-218261e5-cab9-481e-8c4b-54fb48786e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:32.650409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:32.668610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:32.689950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b39556-c207-438d-9014-dbff9aa2c0f2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:32.699341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:32.700593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:32.700965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:32.701315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] PATH_INFO: `/v3/projects/c26a156485b64e4fa9fa10e4a19782d4` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:32.702755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c26a156485b64e4fa9fa10e4a19782d4)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:32.705890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:32.723542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218261e5-cab9-481e-8c4b-54fb48786e3c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:32.728523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650c034-56ca-4a46-bd24-d4b03902fd27 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:32.729733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650c034-56ca-4a46-bd24-d4b03902fd27 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:32.730478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650c034-56ca-4a46-bd24-d4b03902fd27 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:32.730949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650c034-56ca-4a46-bd24-d4b03902fd27 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:32.744938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650c034-56ca-4a46-bd24-d4b03902fd27 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:32.746597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650c034-56ca-4a46-bd24-d4b03902fd27 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:32.767774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 912/1802] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:43:32.778144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c26a156485b64e4fa9fa10e4a19782d4', 'request_id': 'req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:32.787056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c26a156485b64e4fa9fa10e4a19782d4', 'request_id': 'req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:32.800960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650c034-56ca-4a46-bd24-d4b03902fd27 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:32.802641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c650c034-56ca-4a46-bd24-d4b03902fd27 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:32.805607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c26a156485b64e4fa9fa10e4a19782d4', 'request_id': 'req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:32.821339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650c034-56ca-4a46-bd24-d4b03902fd27 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:32.826406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] The token cache is being invalidate because project c26a156485b64e4fa9fa10e4a19782d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:32.826777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:32.828013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:32.830142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 891/1803] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:32 2022] DELETE /identity/v3/projects/c26a156485b64e4fa9fa10e4a19782d4 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:32.833231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:32.833231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:32.905081 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:32.907001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:32.907263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:32.907585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] PATH_INFO: `/v3/projects/93778af5acd341feac357943a18da0c4` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:32.908991 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93778af5acd341feac357943a18da0c4)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:32.911960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:32.922813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24090581-504e-46aa-9d5f-b7ea2d3a4ab8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:32.930667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2faa1db-6e40-48c3-b8cf-e085f06ed7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:32.931321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faa1db-6e40-48c3-b8cf-e085f06ed7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:32.931604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faa1db-6e40-48c3-b8cf-e085f06ed7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:32.931880 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2faa1db-6e40-48c3-b8cf-e085f06ed7fe admin admin] PATH_INFO: `/v3/users/146ab2a5cc104752803e20f04a7a96ba` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:32.933282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faa1db-6e40-48c3-b8cf-e085f06ed7fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=146ab2a5cc104752803e20f04a7a96ba)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:32.935266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2faa1db-6e40-48c3-b8cf-e085f06ed7fe admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:32.991933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93778af5acd341feac357943a18da0c4', 'request_id': 'req-bbc47089-8c5f-4461-bc93-dc6d2ee50353'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.002907 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93778af5acd341feac357943a18da0c4', 'request_id': 'req-bbc47089-8c5f-4461-bc93-dc6d2ee50353'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.006752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a2faa1db-6e40-48c3-b8cf-e085f06ed7fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '146ab2a5cc104752803e20f04a7a96ba'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.008375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a2faa1db-6e40-48c3-b8cf-e085f06ed7fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '146ab2a5cc104752803e20f04a7a96ba'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.009534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93778af5acd341feac357943a18da0c4', 'request_id': 'req-bbc47089-8c5f-4461-bc93-dc6d2ee50353'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.022273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a2faa1db-6e40-48c3-b8cf-e085f06ed7fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '146ab2a5cc104752803e20f04a7a96ba'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.029236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] The token cache is being invalidate because project 93778af5acd341feac357943a18da0c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:33.029530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.030743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.032639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a2faa1db-6e40-48c3-b8cf-e085f06ed7fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '146ab2a5cc104752803e20f04a7a96ba'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.033697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 913/1804] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:32 2022] DELETE /identity/v3/projects/93778af5acd341feac357943a18da0c4 => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:33.034505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:33.038278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:33.053755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a2faa1db-6e40-48c3-b8cf-e085f06ed7fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '146ab2a5cc104752803e20f04a7a96ba'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.056479 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:33.067908 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 892/1805] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:32 2022] DELETE /identity/v3/users/146ab2a5cc104752803e20f04a7a96ba => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:33.069886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3817df-c77d-4934-a87f-b53579b531af None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:33.070264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3817df-c77d-4934-a87f-b53579b531af None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:33.070536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3817df-c77d-4934-a87f-b53579b531af None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:33.085027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 893/1806] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:43:33 2022] GET /identity => generated 270 bytes in 17 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:43:33.104372 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3817df-c77d-4934-a87f-b53579b531af None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:33.106716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5d3817df-c77d-4934-a87f-b53579b531af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:33.117452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:33.139272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc47089-8c5f-4461-bc93-dc6d2ee50353 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:33.196888 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3817df-c77d-4934-a87f-b53579b531af None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:33.201717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c466db1a-fe71-469b-aa75-79cc15c504e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:33.202357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466db1a-fe71-469b-aa75-79cc15c504e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:33.202627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466db1a-fe71-469b-aa75-79cc15c504e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:33.202893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c466db1a-fe71-469b-aa75-79cc15c504e3 admin admin] PATH_INFO: `/v3/users/12dd01aa2f2d471dba466771dcb3b37b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:33.204658 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466db1a-fe71-469b-aa75-79cc15c504e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12dd01aa2f2d471dba466771dcb3b37b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:33.207125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c466db1a-fe71-469b-aa75-79cc15c504e3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:33.236484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139782c9-eb20-4de0-9b2b-f0be0537376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:33.238521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139782c9-eb20-4de0-9b2b-f0be0537376e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:33.240376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139782c9-eb20-4de0-9b2b-f0be0537376e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:33.241303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139782c9-eb20-4de0-9b2b-f0be0537376e admin admin] PATH_INFO: `/v3/users/8084f2796bd84fe19d20db5855aba3ea` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:33.243110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139782c9-eb20-4de0-9b2b-f0be0537376e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8084f2796bd84fe19d20db5855aba3ea)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:33.245214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139782c9-eb20-4de0-9b2b-f0be0537376e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:33.276566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c466db1a-fe71-469b-aa75-79cc15c504e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12dd01aa2f2d471dba466771dcb3b37b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.277412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c466db1a-fe71-469b-aa75-79cc15c504e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12dd01aa2f2d471dba466771dcb3b37b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.289311 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c466db1a-fe71-469b-aa75-79cc15c504e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12dd01aa2f2d471dba466771dcb3b37b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.300199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c466db1a-fe71-469b-aa75-79cc15c504e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12dd01aa2f2d471dba466771dcb3b37b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.306536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-139782c9-eb20-4de0-9b2b-f0be0537376e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8084f2796bd84fe19d20db5855aba3ea'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.316105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-139782c9-eb20-4de0-9b2b-f0be0537376e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8084f2796bd84fe19d20db5855aba3ea'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.316512 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c466db1a-fe71-469b-aa75-79cc15c504e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12dd01aa2f2d471dba466771dcb3b37b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.316768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-139782c9-eb20-4de0-9b2b-f0be0537376e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8084f2796bd84fe19d20db5855aba3ea'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.324267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 894/1807] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:33 2022] DELETE /identity/v3/users/12dd01aa2f2d471dba466771dcb3b37b => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:33.329233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66811fbc-5ba5-4b1a-aa51-58b7d9c5b6e9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:33.329233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66811fbc-5ba5-4b1a-aa51-58b7d9c5b6e9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:33.329474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66811fbc-5ba5-4b1a-aa51-58b7d9c5b6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:33.338844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-139782c9-eb20-4de0-9b2b-f0be0537376e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8084f2796bd84fe19d20db5855aba3ea'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.349912 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-139782c9-eb20-4de0-9b2b-f0be0537376e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8084f2796bd84fe19d20db5855aba3ea'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.351483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-66811fbc-5ba5-4b1a-aa51-58b7d9c5b6e9 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:33.353125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-66811fbc-5ba5-4b1a-aa51-58b7d9c5b6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:33.359427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 914/1808] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:33 2022] DELETE /identity/v3/users/8084f2796bd84fe19d20db5855aba3ea => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:33.362951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2186e3-ebc6-444e-945c-b81d79cf2fa1 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:33.363070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2186e3-ebc6-444e-945c-b81d79cf2fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:33.363453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2186e3-ebc6-444e-945c-b81d79cf2fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:33.397793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-3c2186e3-ebc6-444e-945c-b81d79cf2fa1 None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:33.400885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3c2186e3-ebc6-444e-945c-b81d79cf2fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:33.438212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 895/1809] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:33.449160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66811fbc-5ba5-4b1a-aa51-58b7d9c5b6e9 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:33.450964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-66811fbc-5ba5-4b1a-aa51-58b7d9c5b6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:33.483044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:33.510611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 915/1810] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:43:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:33.580448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66811fbc-5ba5-4b1a-aa51-58b7d9c5b6e9 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:33.642659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04c570b-d982-4547-98cb-83be5a465baa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:33.643971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c570b-d982-4547-98cb-83be5a465baa service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:33.644257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c570b-d982-4547-98cb-83be5a465baa service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:33.644531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04c570b-d982-4547-98cb-83be5a465baa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:33.738205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c570b-d982-4547-98cb-83be5a465baa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:33.739650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04c570b-d982-4547-98cb-83be5a465baa service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:33.757595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 896/1811] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:33.765810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2186e3-ebc6-444e-945c-b81d79cf2fa1 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:33.768966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3c2186e3-ebc6-444e-945c-b81d79cf2fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:33.780824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:33.786062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2186e3-ebc6-444e-945c-b81d79cf2fa1 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:33.808731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:33.811156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:33.811807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:33.812189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] PATH_INFO: `/v3/projects/229592fb38ca4599b8064599148d1162` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:33.815831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=229592fb38ca4599b8064599148d1162)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:33.819496 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:33.899362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '229592fb38ca4599b8064599148d1162', 'request_id': 'req-692f0506-fd7c-4b9f-aa98-147cf859bac9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.913460 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '229592fb38ca4599b8064599148d1162', 'request_id': 'req-692f0506-fd7c-4b9f-aa98-147cf859bac9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.929580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '229592fb38ca4599b8064599148d1162', 'request_id': 'req-692f0506-fd7c-4b9f-aa98-147cf859bac9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.948302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] The token cache is being invalidate because project 229592fb38ca4599b8064599148d1162 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:33.948586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.949614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-692f0506-fd7c-4b9f-aa98-147cf859bac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:33.952376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 916/1812] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:33 2022] DELETE /identity/v3/projects/229592fb38ca4599b8064599148d1162 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:33.959353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1270820-40d1-4694-9bed-1ec8e9a3e25d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:33.959353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1270820-40d1-4694-9bed-1ec8e9a3e25d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:33.959727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1270820-40d1-4694-9bed-1ec8e9a3e25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:33.982983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b1270820-40d1-4694-9bed-1ec8e9a3e25d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:33.984472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b1270820-40d1-4694-9bed-1ec8e9a3e25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.076465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 897/1813] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:34.086183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c582c71-f263-4858-a892-29933d76d5b7 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.086472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c582c71-f263-4858-a892-29933d76d5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.086765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c582c71-f263-4858-a892-29933d76d5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.129000 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-9c582c71-f263-4858-a892-29933d76d5b7 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:34.129000 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9c582c71-f263-4858-a892-29933d76d5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.148436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1270820-40d1-4694-9bed-1ec8e9a3e25d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:34.154725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b1270820-40d1-4694-9bed-1ec8e9a3e25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.167153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 917/1814] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:34.171053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c582c71-f263-4858-a892-29933d76d5b7 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:34.183184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c582c71-f263-4858-a892-29933d76d5b7 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:34.192472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3337a-8b1d-424f-8c4a-807e852bddf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:34.193401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3337a-8b1d-424f-8c4a-807e852bddf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.194280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3337a-8b1d-424f-8c4a-807e852bddf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.194743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c3337a-8b1d-424f-8c4a-807e852bddf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.195651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3337a-8b1d-424f-8c4a-807e852bddf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:34.196200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3337a-8b1d-424f-8c4a-807e852bddf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:34.198904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c3337a-8b1d-424f-8c4a-807e852bddf4 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:34.217030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 918/1815] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:43:34.223081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:34.234458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3337a-8b1d-424f-8c4a-807e852bddf4 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:34.241157 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-02c3337a-8b1d-424f-8c4a-807e852bddf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.274178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c3337a-8b1d-424f-8c4a-807e852bddf4 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:34.279263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:34.279896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.280173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.280434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] PATH_INFO: `/v3/projects/04db4237c4f64f9fb4e26b9f44026e5f` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.281652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=04db4237c4f64f9fb4e26b9f44026e5f)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:34.283527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:34.297969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1270820-40d1-4694-9bed-1ec8e9a3e25d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:34.307108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5b9cb-b410-40ed-a88b-051e5cdd6b9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:34.307716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5b9cb-b410-40ed-a88b-051e5cdd6b9b service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.308092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5b9cb-b410-40ed-a88b-051e5cdd6b9b service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.308474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead5b9cb-b410-40ed-a88b-051e5cdd6b9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.310780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ead5b9cb-b410-40ed-a88b-051e5cdd6b9b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.330467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5b9cb-b410-40ed-a88b-051e5cdd6b9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:34.332497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead5b9cb-b410-40ed-a88b-051e5cdd6b9b service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:34.349178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 898/1816] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:43:34.351049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5b9cb-b410-40ed-a88b-051e5cdd6b9b service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:34.364338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead5b9cb-b410-40ed-a88b-051e5cdd6b9b service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:34.367576 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04db4237c4f64f9fb4e26b9f44026e5f', 'request_id': 'req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:34.382874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04db4237c4f64f9fb4e26b9f44026e5f', 'request_id': 'req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:34.407102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04db4237c4f64f9fb4e26b9f44026e5f', 'request_id': 'req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:34.443248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] The token cache is being invalidate because project 04db4237c4f64f9fb4e26b9f44026e5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:34.444927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:34.449577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:34.467376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 919/1817] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:34 2022] DELETE /identity/v3/projects/04db4237c4f64f9fb4e26b9f44026e5f => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:34.470153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:34.478798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.496099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c498de-b87b-4caf-9704-5262a1a5f121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:34.496775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c498de-b87b-4caf-9704-5262a1a5f121 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.497109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c498de-b87b-4caf-9704-5262a1a5f121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.497381 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c498de-b87b-4caf-9704-5262a1a5f121 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.498016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c498de-b87b-4caf-9704-5262a1a5f121 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:34.499992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c498de-b87b-4caf-9704-5262a1a5f121 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:34.568731 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 899/1818] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:34 2022] POST /identity/v3/projects => generated 385 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:34.571464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c08d80-06be-4352-a1e8-905e855a4a9f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.572128 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c08d80-06be-4352-a1e8-905e855a4a9f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.573029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c08d80-06be-4352-a1e8-905e855a4a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.583348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:34.601502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd6d34f-be0d-4b19-a0fc-af725919ed8a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:34.606396 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-61c08d80-06be-4352-a1e8-905e855a4a9f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:34.609427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-61c08d80-06be-4352-a1e8-905e855a4a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.683750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:34.684349 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.684808 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.685111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] PATH_INFO: `/v3/projects/98f6a915c6a8400e9effc414b3e526d2` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.686530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98f6a915c6a8400e9effc414b3e526d2)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:34.688753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:34.728986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 900/1819] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:34.730140 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c08d80-06be-4352-a1e8-905e855a4a9f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:34.731998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-61c08d80-06be-4352-a1e8-905e855a4a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.777630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c08d80-06be-4352-a1e8-905e855a4a9f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:34.785739 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98f6a915c6a8400e9effc414b3e526d2', 'request_id': 'req-8c57a04b-eaef-49de-8955-81dd905a5d01'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:34.799962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98f6a915c6a8400e9effc414b3e526d2', 'request_id': 'req-8c57a04b-eaef-49de-8955-81dd905a5d01'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:34.807570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98f6a915c6a8400e9effc414b3e526d2', 'request_id': 'req-8c57a04b-eaef-49de-8955-81dd905a5d01'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:34.823097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] The token cache is being invalidate because project 98f6a915c6a8400e9effc414b3e526d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:34.823381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:34.824751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8c57a04b-eaef-49de-8955-81dd905a5d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:34.826918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 920/1820] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:34 2022] DELETE /identity/v3/projects/98f6a915c6a8400e9effc414b3e526d2 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:34.828147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18d52-7269-43ec-9ba7-aacc58136963 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.829754 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18d52-7269-43ec-9ba7-aacc58136963 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.830036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede18d52-7269-43ec-9ba7-aacc58136963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.836380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:34.857362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-ede18d52-7269-43ec-9ba7-aacc58136963 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:34.859209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ede18d52-7269-43ec-9ba7-aacc58136963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.876963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc80323-33e5-447f-b8dc-3ac5f6caae53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:34.877581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80323-33e5-447f-b8dc-3ac5f6caae53 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.877894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80323-33e5-447f-b8dc-3ac5f6caae53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.878163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc80323-33e5-447f-b8dc-3ac5f6caae53 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.878788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc80323-33e5-447f-b8dc-3ac5f6caae53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:34.880711 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc80323-33e5-447f-b8dc-3ac5f6caae53 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:34.903661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 901/1821] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:34 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:34.905040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8dbb0-d805-400e-a3c5-c5e109ef7bd7 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.905381 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8dbb0-d805-400e-a3c5-c5e109ef7bd7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.905658 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc8dbb0-d805-400e-a3c5-c5e109ef7bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.926083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-bbc8dbb0-d805-400e-a3c5-c5e109ef7bd7 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:34.927286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 921/1822] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:34.927429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bbc8dbb0-d805-400e-a3c5-c5e109ef7bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.928235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede18d52-7269-43ec-9ba7-aacc58136963 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:34.929476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ede18d52-7269-43ec-9ba7-aacc58136963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.935492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 902/1823] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:34.936478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8dbb0-d805-400e-a3c5-c5e109ef7bd7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:34.945590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede18d52-7269-43ec-9ba7-aacc58136963 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:34.947330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc8dbb0-d805-400e-a3c5-c5e109ef7bd7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:34.950685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a429701f-ad8d-4f7a-bdd9-1b262bf16332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:34.951500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429701f-ad8d-4f7a-bdd9-1b262bf16332 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.951744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429701f-ad8d-4f7a-bdd9-1b262bf16332 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.952053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429701f-ad8d-4f7a-bdd9-1b262bf16332 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.952716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429701f-ad8d-4f7a-bdd9-1b262bf16332 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:34.953084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171216d-f0e0-4ac9-88b0-20e8cfd65f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:34.954435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171216d-f0e0-4ac9-88b0-20e8cfd65f5d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.954897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171216d-f0e0-4ac9-88b0-20e8cfd65f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.955012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a429701f-ad8d-4f7a-bdd9-1b262bf16332 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:34.955511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a171216d-f0e0-4ac9-88b0-20e8cfd65f5d admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.956541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171216d-f0e0-4ac9-88b0-20e8cfd65f5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:34.957061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171216d-f0e0-4ac9-88b0-20e8cfd65f5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:34.959281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a171216d-f0e0-4ac9-88b0-20e8cfd65f5d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:34.965483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 922/1824] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:34.966783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e1574-9220-4d9b-92e9-6c96082cd63d None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.967065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e1574-9220-4d9b-92e9-6c96082cd63d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.967334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248e1574-9220-4d9b-92e9-6c96082cd63d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.975347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 903/1825] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:43:34.976414 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171216d-f0e0-4ac9-88b0-20e8cfd65f5d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:34.989671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-248e1574-9220-4d9b-92e9-6c96082cd63d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:34.990133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a171216d-f0e0-4ac9-88b0-20e8cfd65f5d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:34.990873 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-248e1574-9220-4d9b-92e9-6c96082cd63d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:34.995732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f589df-cfea-47da-a414-89d1a9269349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:34.996548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f589df-cfea-47da-a414-89d1a9269349 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:34.997057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f589df-cfea-47da-a414-89d1a9269349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:34.997599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f589df-cfea-47da-a414-89d1a9269349 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:34.998433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f589df-cfea-47da-a414-89d1a9269349 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:34.999244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 923/1826] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:35.000088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e1574-9220-4d9b-92e9-6c96082cd63d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.000916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f589df-cfea-47da-a414-89d1a9269349 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.014028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 904/1827] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:35.018331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f589df-cfea-47da-a414-89d1a9269349 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.021116 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248e1574-9220-4d9b-92e9-6c96082cd63d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.028032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537003e-0b39-4b1d-85b1-1cb48709d3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.028931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537003e-0b39-4b1d-85b1-1cb48709d3cb admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.029414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537003e-0b39-4b1d-85b1-1cb48709d3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.029693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7537003e-0b39-4b1d-85b1-1cb48709d3cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.030324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537003e-0b39-4b1d-85b1-1cb48709d3cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.033215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f589df-cfea-47da-a414-89d1a9269349 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.033768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7537003e-0b39-4b1d-85b1-1cb48709d3cb admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.039163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df796d-2ae3-4fd1-856f-72239786bdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.039707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df796d-2ae3-4fd1-856f-72239786bdca admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.040425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df796d-2ae3-4fd1-856f-72239786bdca admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.040425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32df796d-2ae3-4fd1-856f-72239786bdca admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.041362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df796d-2ae3-4fd1-856f-72239786bdca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:35.041945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df796d-2ae3-4fd1-856f-72239786bdca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.044310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32df796d-2ae3-4fd1-856f-72239786bdca admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.044792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:35.071940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 905/1828] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:43:35.073167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df796d-2ae3-4fd1-856f-72239786bdca admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.074649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 924/1829] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:34 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:35.075547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537003e-0b39-4b1d-85b1-1cb48709d3cb admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.090359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32df796d-2ae3-4fd1-856f-72239786bdca admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.154142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7537003e-0b39-4b1d-85b1-1cb48709d3cb admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.162807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6105e0-0dd4-493d-856d-20ddbcc41b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.163608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6105e0-0dd4-493d-856d-20ddbcc41b54 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.163967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6105e0-0dd4-493d-856d-20ddbcc41b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.164360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6105e0-0dd4-493d-856d-20ddbcc41b54 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.170552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65143b7-253c-4e26-a6c8-434fe2014919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.171380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65143b7-253c-4e26-a6c8-434fe2014919 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.171770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65143b7-253c-4e26-a6c8-434fe2014919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.173251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65143b7-253c-4e26-a6c8-434fe2014919 admin admin] PATH_INFO: `/v3/projects/792ddd4261914ab5adb4224535cde27f/users/6ee3ef8dda444defb1875513f6700c50/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.176337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6105e0-0dd4-493d-856d-20ddbcc41b54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.180338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6105e0-0dd4-493d-856d-20ddbcc41b54 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.191970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65143b7-253c-4e26-a6c8-434fe2014919 admin admin] RBAC: Authorizing `identity:create_grant(project_id=792ddd4261914ab5adb4224535cde27f, user_id=6ee3ef8dda444defb1875513f6700c50, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.194974 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b65143b7-253c-4e26-a6c8-434fe2014919 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.216813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 906/1830] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:35 2022] PUT /identity/v3/projects/792ddd4261914ab5adb4224535cde27f/users/6ee3ef8dda444defb1875513f6700c50/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:35.217780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65143b7-253c-4e26-a6c8-434fe2014919 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.237015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 925/1831] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:35 2022] POST /identity/v3/users => generated 293 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:35.238492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635e8ff-0052-4b59-81cc-d07369975a6e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.238788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635e8ff-0052-4b59-81cc-d07369975a6e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.239989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c635e8ff-0052-4b59-81cc-d07369975a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.288987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c635e8ff-0052-4b59-81cc-d07369975a6e None None] MFA Rules not processed for user `6ee3ef8dda444defb1875513f6700c50`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:35.290350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c635e8ff-0052-4b59-81cc-d07369975a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:35.326180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65143b7-253c-4e26-a6c8-434fe2014919 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.335442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec33e3-9daa-496f-8bd4-1d954d283323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.336075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec33e3-9daa-496f-8bd4-1d954d283323 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.336372 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec33e3-9daa-496f-8bd4-1d954d283323 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.336621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ec33e3-9daa-496f-8bd4-1d954d283323 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.337295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec33e3-9daa-496f-8bd4-1d954d283323 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.339225 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ec33e3-9daa-496f-8bd4-1d954d283323 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.362781 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:35.398571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 926/1832] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:35.402027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635e8ff-0052-4b59-81cc-d07369975a6e None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.402145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 907/1833] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:35 2022] POST /identity/v3/projects => generated 373 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:35.402881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec33e3-9daa-496f-8bd4-1d954d283323 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.413971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c635e8ff-0052-4b59-81cc-d07369975a6e None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.485276 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed3456-2dd3-4be6-a913-e1a1782310c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.486231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed3456-2dd3-4be6-a913-e1a1782310c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.486664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed3456-2dd3-4be6-a913-e1a1782310c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.487109 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ed3456-2dd3-4be6-a913-e1a1782310c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.488093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed3456-2dd3-4be6-a913-e1a1782310c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.491461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ed3456-2dd3-4be6-a913-e1a1782310c3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.503849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ec33e3-9daa-496f-8bd4-1d954d283323 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.504313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 927/1834] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:35.508105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed3456-2dd3-4be6-a913-e1a1782310c3 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.509783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0b7d9-1bf1-41dd-b57d-74e84e6869f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.510402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0b7d9-1bf1-41dd-b57d-74e84e6869f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.510674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0b7d9-1bf1-41dd-b57d-74e84e6869f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.510940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea0b7d9-1bf1-41dd-b57d-74e84e6869f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.511784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0b7d9-1bf1-41dd-b57d-74e84e6869f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.513763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea0b7d9-1bf1-41dd-b57d-74e84e6869f5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.514595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:35.520868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ed3456-2dd3-4be6-a913-e1a1782310c3 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.532594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e53e26-e737-4230-ae56-b77be10b4a18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.533257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e53e26-e737-4230-ae56-b77be10b4a18 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.533535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e53e26-e737-4230-ae56-b77be10b4a18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.533807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e53e26-e737-4230-ae56-b77be10b4a18 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.534658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e53e26-e737-4230-ae56-b77be10b4a18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.536381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e53e26-e737-4230-ae56-b77be10b4a18 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.591845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 908/1835] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:35 2022] POST /identity/v3/projects => generated 385 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:35.593151 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0b7d9-1bf1-41dd-b57d-74e84e6869f5 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.598190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 928/1836] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:35 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:35.600268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e53e26-e737-4230-ae56-b77be10b4a18 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.613331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e53e26-e737-4230-ae56-b77be10b4a18 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.678566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb82d0c-2123-4e27-8431-9ab8165c8a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.679503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb82d0c-2123-4e27-8431-9ab8165c8a1f admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.679785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb82d0c-2123-4e27-8431-9ab8165c8a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.680040 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb82d0c-2123-4e27-8431-9ab8165c8a1f admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.681464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb82d0c-2123-4e27-8431-9ab8165c8a1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.682655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb82d0c-2123-4e27-8431-9ab8165c8a1f admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.684581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea0b7d9-1bf1-41dd-b57d-74e84e6869f5 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.690520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee8d7b-236a-491d-93d7-e3db9f580238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.691145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee8d7b-236a-491d-93d7-e3db9f580238 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.691429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee8d7b-236a-491d-93d7-e3db9f580238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.691702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee8d7b-236a-491d-93d7-e3db9f580238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.692352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee8d7b-236a-491d-93d7-e3db9f580238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.693992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee8d7b-236a-491d-93d7-e3db9f580238 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.708104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 909/1837] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:35.709151 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee8d7b-236a-491d-93d7-e3db9f580238 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.714756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 929/1838] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:35 2022] POST /identity/v3/users => generated 299 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:35.714756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb82d0c-2123-4e27-8431-9ab8165c8a1f admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.722977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee8d7b-236a-491d-93d7-e3db9f580238 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.724966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb82d0c-2123-4e27-8431-9ab8165c8a1f admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.729681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787ec5c-285f-4910-9c17-3edd9fa04a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.730280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8f3ce-84a0-4112-af7a-4b9619d85e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.730382 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787ec5c-285f-4910-9c17-3edd9fa04a83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.730604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787ec5c-285f-4910-9c17-3edd9fa04a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.730825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8f3ce-84a0-4112-af7a-4b9619d85e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.730926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787ec5c-285f-4910-9c17-3edd9fa04a83 admin admin] PATH_INFO: `/v3/projects/6c94dcf829b74a6cb9c1c6ba2cbfeb75/users/853cbe4820d74ecc89bf2ab116b55ae3/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.731099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8f3ce-84a0-4112-af7a-4b9619d85e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.731368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc8f3ce-84a0-4112-af7a-4b9619d85e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.732191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8f3ce-84a0-4112-af7a-4b9619d85e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.736239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc8f3ce-84a0-4112-af7a-4b9619d85e2d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.747518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787ec5c-285f-4910-9c17-3edd9fa04a83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c94dcf829b74a6cb9c1c6ba2cbfeb75, user_id=853cbe4820d74ecc89bf2ab116b55ae3, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.750786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d787ec5c-285f-4910-9c17-3edd9fa04a83 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.753349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 910/1839] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:35.754598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8f3ce-84a0-4112-af7a-4b9619d85e2d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.765325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc8f3ce-84a0-4112-af7a-4b9619d85e2d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.768561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 930/1840] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:35 2022] PUT /identity/v3/projects/6c94dcf829b74a6cb9c1c6ba2cbfeb75/users/853cbe4820d74ecc89bf2ab116b55ae3/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:35.769761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787ec5c-285f-4910-9c17-3edd9fa04a83 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.842504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1a569c-4669-4646-8530-c878a6a135e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.843109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a569c-4669-4646-8530-c878a6a135e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.843380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a569c-4669-4646-8530-c878a6a135e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.843642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1a569c-4669-4646-8530-c878a6a135e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.844944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a569c-4669-4646-8530-c878a6a135e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.846095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d787ec5c-285f-4910-9c17-3edd9fa04a83 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.846505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1a569c-4669-4646-8530-c878a6a135e7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.851155 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53391b-6669-4a69-b2a9-9b2ba21354ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.851859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53391b-6669-4a69-b2a9-9b2ba21354ef admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.852204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53391b-6669-4a69-b2a9-9b2ba21354ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.852539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a53391b-6669-4a69-b2a9-9b2ba21354ef admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.853284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53391b-6669-4a69-b2a9-9b2ba21354ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.854218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:35.855264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a53391b-6669-4a69-b2a9-9b2ba21354ef admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.867475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 911/1841] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:35.867475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e4731-fcd1-4b19-8902-2a5f59bd23b0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.867475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e4731-fcd1-4b19-8902-2a5f59bd23b0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.867475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94e4731-fcd1-4b19-8902-2a5f59bd23b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.871897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 931/1842] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:35.872828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53391b-6669-4a69-b2a9-9b2ba21354ef admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.883181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a53391b-6669-4a69-b2a9-9b2ba21354ef admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:35.889743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfe9ea-6202-4bb6-8b91-944a5555ae50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:35.890412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfe9ea-6202-4bb6-8b91-944a5555ae50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:35.890885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfe9ea-6202-4bb6-8b91-944a5555ae50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:35.891128 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecfe9ea-6202-4bb6-8b91-944a5555ae50 admin admin] PATH_INFO: `/v3/projects/ae24b58d42174abeb14c231c5baada1c/users/0247e19b972644eaa23d1cb0b0e70b25/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:35.906343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c94e4731-fcd1-4b19-8902-2a5f59bd23b0 None None] MFA Rules not processed for user `853cbe4820d74ecc89bf2ab116b55ae3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:35.908063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c94e4731-fcd1-4b19-8902-2a5f59bd23b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:35.908480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfe9ea-6202-4bb6-8b91-944a5555ae50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae24b58d42174abeb14c231c5baada1c, user_id=0247e19b972644eaa23d1cb0b0e70b25, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:35.911570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecfe9ea-6202-4bb6-8b91-944a5555ae50 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:35.928730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 932/1843] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:35 2022] PUT /identity/v3/projects/ae24b58d42174abeb14c231c5baada1c/users/0247e19b972644eaa23d1cb0b0e70b25/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:35.929764 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfe9ea-6202-4bb6-8b91-944a5555ae50 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.962127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:35.985925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 912/1844] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:35.986748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e4731-fcd1-4b19-8902-2a5f59bd23b0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:35.997837 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94e4731-fcd1-4b19-8902-2a5f59bd23b0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.013824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecfe9ea-6202-4bb6-8b91-944a5555ae50 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.020165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc8125-37df-451c-9aaf-b874159f6083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.021396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc8125-37df-451c-9aaf-b874159f6083 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.021898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc8125-37df-451c-9aaf-b874159f6083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.022392 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedc8125-37df-451c-9aaf-b874159f6083 admin admin] PATH_INFO: `/v3/projects/c8e315b64f8b4a34ad9ce4f884617efb/users/c665c93c2b0941f3860d0e51cd48c63e/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.040515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc8125-37df-451c-9aaf-b874159f6083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e315b64f8b4a34ad9ce4f884617efb, user_id=c665c93c2b0941f3860d0e51cd48c63e, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.045075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedc8125-37df-451c-9aaf-b874159f6083 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.062195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 933/1845] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:35 2022] PUT /identity/v3/projects/c8e315b64f8b4a34ad9ce4f884617efb/users/c665c93c2b0941f3860d0e51cd48c63e/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:36.063315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac010f08-8678-4f4c-be0a-ee5b332055d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.063940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac010f08-8678-4f4c-be0a-ee5b332055d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.064212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac010f08-8678-4f4c-be0a-ee5b332055d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.064477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac010f08-8678-4f4c-be0a-ee5b332055d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.065177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac010f08-8678-4f4c-be0a-ee5b332055d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.065546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc8125-37df-451c-9aaf-b874159f6083 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:36.066789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac010f08-8678-4f4c-be0a-ee5b332055d7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.085523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 913/1846] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:35 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:36.085523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823733ab-2f6b-4964-ae52-f5711fd7182c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.085523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823733ab-2f6b-4964-ae52-f5711fd7182c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.085523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823733ab-2f6b-4964-ae52-f5711fd7182c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.117081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-823733ab-2f6b-4964-ae52-f5711fd7182c None None] MFA Rules not processed for user `c665c93c2b0941f3860d0e51cd48c63e`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:36.118283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-823733ab-2f6b-4964-ae52-f5711fd7182c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:36.144557 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedc8125-37df-451c-9aaf-b874159f6083 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.150170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41f8c4-735a-4ab1-883a-98a94e12687d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.150932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41f8c4-735a-4ab1-883a-98a94e12687d admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.151329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41f8c4-735a-4ab1-883a-98a94e12687d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.151677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41f8c4-735a-4ab1-883a-98a94e12687d admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.152401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41f8c4-735a-4ab1-883a-98a94e12687d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.154462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41f8c4-735a-4ab1-883a-98a94e12687d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.188605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 934/1847] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:36 2022] POST /identity/v3/projects => generated 387 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:36.189693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41f8c4-735a-4ab1-883a-98a94e12687d admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:36.199081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 914/1848] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:36.202426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823733ab-2f6b-4964-ae52-f5711fd7182c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:36.216487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823733ab-2f6b-4964-ae52-f5711fd7182c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.225228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:36.252501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41f8c4-735a-4ab1-883a-98a94e12687d admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.257154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb72c6-ccdc-4d16-bf75-11b8a90c891e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.257793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb72c6-ccdc-4d16-bf75-11b8a90c891e admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.258066 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb72c6-ccdc-4d16-bf75-11b8a90c891e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.258386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb72c6-ccdc-4d16-bf75-11b8a90c891e admin admin] PATH_INFO: `/v3/projects/ae24b58d42174abeb14c231c5baada1c/users/0247e19b972644eaa23d1cb0b0e70b25/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.260459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb72c6-ccdc-4d16-bf75-11b8a90c891e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae24b58d42174abeb14c231c5baada1c, user_id=0247e19b972644eaa23d1cb0b0e70b25, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.264072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb72c6-ccdc-4d16-bf75-11b8a90c891e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.275210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 935/1849] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:36 2022] PUT /identity/v3/projects/ae24b58d42174abeb14c231c5baada1c/users/0247e19b972644eaa23d1cb0b0e70b25/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:36.276042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb72c6-ccdc-4d16-bf75-11b8a90c891e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:36.276964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53b65b-f3d1-49f1-82c6-76ae2e9dabb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.277580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53b65b-f3d1-49f1-82c6-76ae2e9dabb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.277703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2dfb72c6-ccdc-4d16-bf75-11b8a90c891e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:36.277857 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53b65b-f3d1-49f1-82c6-76ae2e9dabb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.278124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53b65b-f3d1-49f1-82c6-76ae2e9dabb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.278827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53b65b-f3d1-49f1-82c6-76ae2e9dabb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.280697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53b65b-f3d1-49f1-82c6-76ae2e9dabb4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.305589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 915/1850] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:36 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:36.305589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05f651-7698-4632-93b5-9d130fd7e8b2 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.305589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05f651-7698-4632-93b5-9d130fd7e8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.305589 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c05f651-7698-4632-93b5-9d130fd7e8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.317794 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:36.333641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2c05f651-7698-4632-93b5-9d130fd7e8b2 None None] MFA Rules not processed for user `0247e19b972644eaa23d1cb0b0e70b25`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:36.334868 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2c05f651-7698-4632-93b5-9d130fd7e8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:36.371474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb72c6-ccdc-4d16-bf75-11b8a90c891e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.375209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37773870-fa1b-40c1-b6e6-ef1f66a41acd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42cf265450454ff9a27a29093336205e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.375856 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37773870-fa1b-40c1-b6e6-ef1f66a41acd service glance] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.376137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37773870-fa1b-40c1-b6e6-ef1f66a41acd service glance] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.376406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37773870-fa1b-40c1-b6e6-ef1f66a41acd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.408258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 916/1851] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:36.412654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05f651-7698-4632-93b5-9d130fd7e8b2 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:36.435059 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37773870-fa1b-40c1-b6e6-ef1f66a41acd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.436515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37773870-fa1b-40c1-b6e6-ef1f66a41acd service glance] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.445980 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 936/1852] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:36 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:36.447293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37773870-fa1b-40c1-b6e6-ef1f66a41acd service glance] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:36.448887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-37773870-fa1b-40c1-b6e6-ef1f66a41acd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:36.489728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c05f651-7698-4632-93b5-9d130fd7e8b2 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.495650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d1811-16e3-4b99-87d1-4df92ebd0074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.496346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d1811-16e3-4b99-87d1-4df92ebd0074 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.496619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d1811-16e3-4b99-87d1-4df92ebd0074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.496908 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d1811-16e3-4b99-87d1-4df92ebd0074 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.497543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d1811-16e3-4b99-87d1-4df92ebd0074 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.499122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d1811-16e3-4b99-87d1-4df92ebd0074 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.513830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 917/1853] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:36.513974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37773870-fa1b-40c1-b6e6-ef1f66a41acd service glance] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.514641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d1811-16e3-4b99-87d1-4df92ebd0074 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:36.515810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525ab6a-e268-42d1-a912-3dbd17127ad9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.517001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525ab6a-e268-42d1-a912-3dbd17127ad9 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.517001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525ab6a-e268-42d1-a912-3dbd17127ad9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.517291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b525ab6a-e268-42d1-a912-3dbd17127ad9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.525762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d1811-16e3-4b99-87d1-4df92ebd0074 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.528863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525ab6a-e268-42d1-a912-3dbd17127ad9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.530659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b525ab6a-e268-42d1-a912-3dbd17127ad9 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.530778 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6c02a-d300-4dfb-8ca5-64b33c33327e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.532060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6c02a-d300-4dfb-8ca5-64b33c33327e admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.532560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6c02a-d300-4dfb-8ca5-64b33c33327e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.532874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc6c02a-d300-4dfb-8ca5-64b33c33327e admin admin] PATH_INFO: `/v3/projects/93d751d7d1424c3a83c6c663d6dffc8f/users/c06738a789ec46e69790e093678f380b/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.541958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 937/1854] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:36 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:36.554329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6c02a-d300-4dfb-8ca5-64b33c33327e admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d751d7d1424c3a83c6c663d6dffc8f, user_id=c06738a789ec46e69790e093678f380b, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.558227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc6c02a-d300-4dfb-8ca5-64b33c33327e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.571798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:36.576564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 918/1855] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:36 2022] PUT /identity/v3/projects/93d751d7d1424c3a83c6c663d6dffc8f/users/c06738a789ec46e69790e093678f380b/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:36.581549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525ab6a-e268-42d1-a912-3dbd17127ad9 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:36.609303 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6c02a-d300-4dfb-8ca5-64b33c33327e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:36.638560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b525ab6a-e268-42d1-a912-3dbd17127ad9 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.724686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc6c02a-d300-4dfb-8ca5-64b33c33327e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.735595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4541b0a6-293a-4697-8600-34ea2085e755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.736727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b0a6-293a-4697-8600-34ea2085e755 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.737095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b0a6-293a-4697-8600-34ea2085e755 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.737434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4541b0a6-293a-4697-8600-34ea2085e755 admin admin] PATH_INFO: `/v3/users/6ee3ef8dda444defb1875513f6700c50` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.739064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541b0a6-293a-4697-8600-34ea2085e755 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6ee3ef8dda444defb1875513f6700c50)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.741221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4541b0a6-293a-4697-8600-34ea2085e755 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.745225 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a14be7-b53a-410f-80bf-6f2967a6290d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.745843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a14be7-b53a-410f-80bf-6f2967a6290d admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.746125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a14be7-b53a-410f-80bf-6f2967a6290d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.746645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a14be7-b53a-410f-80bf-6f2967a6290d admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.747300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a14be7-b53a-410f-80bf-6f2967a6290d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.749082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a14be7-b53a-410f-80bf-6f2967a6290d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.755950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:36.774644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 938/1856] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:36 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:36.775718 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a14be7-b53a-410f-80bf-6f2967a6290d admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:36.788209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a14be7-b53a-410f-80bf-6f2967a6290d admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:36.797046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb4e929-100c-4369-96f4-04d5d64eb3ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:36.798240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4e929-100c-4369-96f4-04d5d64eb3ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.798586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4e929-100c-4369-96f4-04d5d64eb3ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.798927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb4e929-100c-4369-96f4-04d5d64eb3ae admin admin] PATH_INFO: `/v3/projects/93d751d7d1424c3a83c6c663d6dffc8f/users/c06738a789ec46e69790e093678f380b/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.801817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4e929-100c-4369-96f4-04d5d64eb3ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=93d751d7d1424c3a83c6c663d6dffc8f, user_id=c06738a789ec46e69790e093678f380b, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:36.805240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb4e929-100c-4369-96f4-04d5d64eb3ae admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:36.821965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 939/1857] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:36 2022] PUT /identity/v3/projects/93d751d7d1424c3a83c6c663d6dffc8f/users/c06738a789ec46e69790e093678f380b/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:36.834569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d38bce-6b04-46fb-b5aa-2be46e35b1c8 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.835599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d38bce-6b04-46fb-b5aa-2be46e35b1c8 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.835599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d38bce-6b04-46fb-b5aa-2be46e35b1c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.850305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4541b0a6-293a-4697-8600-34ea2085e755 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6ee3ef8dda444defb1875513f6700c50'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:36.851160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4541b0a6-293a-4697-8600-34ea2085e755 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6ee3ef8dda444defb1875513f6700c50'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:36.866559 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4541b0a6-293a-4697-8600-34ea2085e755 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6ee3ef8dda444defb1875513f6700c50'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:36.909993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4541b0a6-293a-4697-8600-34ea2085e755 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6ee3ef8dda444defb1875513f6700c50'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:36.913515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-19d38bce-6b04-46fb-b5aa-2be46e35b1c8 None None] MFA Rules not processed for user `c06738a789ec46e69790e093678f380b`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:36.914884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-19d38bce-6b04-46fb-b5aa-2be46e35b1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:36.927575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4541b0a6-293a-4697-8600-34ea2085e755 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6ee3ef8dda444defb1875513f6700c50'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:36.937115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 919/1858] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:36 2022] DELETE /identity/v3/users/6ee3ef8dda444defb1875513f6700c50 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:36.941997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20061879-9d6b-4696-ab38-985a1989bf86 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:36.942459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20061879-9d6b-4696-ab38-985a1989bf86 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:36.942895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20061879-9d6b-4696-ab38-985a1989bf86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:36.975511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-20061879-9d6b-4696-ab38-985a1989bf86 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:36.978354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-20061879-9d6b-4696-ab38-985a1989bf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:37.028809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 940/1859] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:36 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:37.038194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d38bce-6b04-46fb-b5aa-2be46e35b1c8 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:37.039636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-19d38bce-6b04-46fb-b5aa-2be46e35b1c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:37.088084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 920/1860] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:37.105428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20061879-9d6b-4696-ab38-985a1989bf86 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:37.110349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-20061879-9d6b-4696-ab38-985a1989bf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:37.124988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d38bce-6b04-46fb-b5aa-2be46e35b1c8 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:37.153329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:37.184287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230143c8-bd57-4026-951c-1624db1edf09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:37.185352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230143c8-bd57-4026-951c-1624db1edf09 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:37.186079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230143c8-bd57-4026-951c-1624db1edf09 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:37.186682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230143c8-bd57-4026-951c-1624db1edf09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:37.205638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20061879-9d6b-4696-ab38-985a1989bf86 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:37.211018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492890f4-4486-40ff-ba8e-e12f4c290d39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:37.214232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492890f4-4486-40ff-ba8e-e12f4c290d39 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:37.215026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492890f4-4486-40ff-ba8e-e12f4c290d39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:37.215325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492890f4-4486-40ff-ba8e-e12f4c290d39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:37.229810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492890f4-4486-40ff-ba8e-e12f4c290d39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:37.231348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492890f4-4486-40ff-ba8e-e12f4c290d39 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:37.244504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 921/1861] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:43:37.254711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:37.282570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230143c8-bd57-4026-951c-1624db1edf09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:37.284445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-230143c8-bd57-4026-951c-1624db1edf09 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:37.301029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 941/1862] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:37 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:37.592436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492890f4-4486-40ff-ba8e-e12f4c290d39 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:37.594057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-492890f4-4486-40ff-ba8e-e12f4c290d39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:37.608849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492890f4-4486-40ff-ba8e-e12f4c290d39 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:37.614444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:37.615421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:37.615928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:37.616400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] PATH_INFO: `/v3/projects/792ddd4261914ab5adb4224535cde27f` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:37.617886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=792ddd4261914ab5adb4224535cde27f)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:37.619803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:37.676674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '792ddd4261914ab5adb4224535cde27f', 'request_id': 'req-b319e975-f903-4c46-b817-096a25cb5b2c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:37.683163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '792ddd4261914ab5adb4224535cde27f', 'request_id': 'req-b319e975-f903-4c46-b817-096a25cb5b2c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:37.691400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '792ddd4261914ab5adb4224535cde27f', 'request_id': 'req-b319e975-f903-4c46-b817-096a25cb5b2c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:37.702500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] The token cache is being invalidate because project 792ddd4261914ab5adb4224535cde27f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:37.703073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:37.704354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:37.705883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 922/1863] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:37 2022] DELETE /identity/v3/projects/792ddd4261914ab5adb4224535cde27f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:37.710389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230143c8-bd57-4026-951c-1624db1edf09 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:37.722039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230143c8-bd57-4026-951c-1624db1edf09 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:37.789311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afce2a3-5899-43c3-ac3c-5be448392c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:37.789943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce2a3-5899-43c3-ac3c-5be448392c65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:37.790227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce2a3-5899-43c3-ac3c-5be448392c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:37.790505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afce2a3-5899-43c3-ac3c-5be448392c65 admin admin] PATH_INFO: `/v3/users/0247e19b972644eaa23d1cb0b0e70b25` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:37.791633 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce2a3-5899-43c3-ac3c-5be448392c65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0247e19b972644eaa23d1cb0b0e70b25)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:37.793733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afce2a3-5899-43c3-ac3c-5be448392c65 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:37.847851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2afce2a3-5899-43c3-ac3c-5be448392c65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0247e19b972644eaa23d1cb0b0e70b25'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:37.858132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2afce2a3-5899-43c3-ac3c-5be448392c65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0247e19b972644eaa23d1cb0b0e70b25'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:37.858803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2afce2a3-5899-43c3-ac3c-5be448392c65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0247e19b972644eaa23d1cb0b0e70b25'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:37.874664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2afce2a3-5899-43c3-ac3c-5be448392c65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0247e19b972644eaa23d1cb0b0e70b25'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:37.886074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2afce2a3-5899-43c3-ac3c-5be448392c65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0247e19b972644eaa23d1cb0b0e70b25'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:37.888016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 942/1864] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:37 2022] DELETE /identity/v3/users/0247e19b972644eaa23d1cb0b0e70b25 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:37.982812 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:37.985894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:38.002598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b319e975-f903-4c46-b817-096a25cb5b2c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:38.077879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:38.078928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:38.079174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:38.079441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] PATH_INFO: `/v3/users/853cbe4820d74ecc89bf2ab116b55ae3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:38.080597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=853cbe4820d74ecc89bf2ab116b55ae3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:38.082681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:38.140453 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '853cbe4820d74ecc89bf2ab116b55ae3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:38.141306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '853cbe4820d74ecc89bf2ab116b55ae3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:38.143420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:38.165244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '853cbe4820d74ecc89bf2ab116b55ae3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:38.173901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '853cbe4820d74ecc89bf2ab116b55ae3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:38.191236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '853cbe4820d74ecc89bf2ab116b55ae3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:38.207768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 923/1865] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:37 2022] DELETE /identity/v3/users/853cbe4820d74ecc89bf2ab116b55ae3 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:38.210480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbee0d-7f6f-428f-a21e-a5e0c9f72880 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:38.211088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbee0d-7f6f-428f-a21e-a5e0c9f72880 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:38.211499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fbee0d-7f6f-428f-a21e-a5e0c9f72880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:38.233864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:38.236419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:38.259525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-88fbee0d-7f6f-428f-a21e-a5e0c9f72880 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:38.260838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-88fbee0d-7f6f-428f-a21e-a5e0c9f72880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:38.331874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:38.346585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 943/1866] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:38.356255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbee0d-7f6f-428f-a21e-a5e0c9f72880 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:38.358579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-88fbee0d-7f6f-428f-a21e-a5e0c9f72880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:38.384306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9958be-9f47-4adb-a3c2-2377ae191d7a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:38.389347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99865adc-2127-4a19-b745-998955786847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:38.390369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99865adc-2127-4a19-b745-998955786847 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:38.390648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99865adc-2127-4a19-b745-998955786847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:38.390968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99865adc-2127-4a19-b745-998955786847 admin admin] PATH_INFO: `/v3/projects/ae24b58d42174abeb14c231c5baada1c` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:38.392165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99865adc-2127-4a19-b745-998955786847 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae24b58d42174abeb14c231c5baada1c)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:38.394368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99865adc-2127-4a19-b745-998955786847 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:38.430828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fbee0d-7f6f-428f-a21e-a5e0c9f72880 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:38.438988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037ab3ea-1188-4e73-8f49-f83c6b034c98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:38.439703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ab3ea-1188-4e73-8f49-f83c6b034c98 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:38.439985 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ab3ea-1188-4e73-8f49-f83c6b034c98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:38.440256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037ab3ea-1188-4e73-8f49-f83c6b034c98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:38.458463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ab3ea-1188-4e73-8f49-f83c6b034c98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:38.458891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99865adc-2127-4a19-b745-998955786847 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae24b58d42174abeb14c231c5baada1c', 'request_id': 'req-99865adc-2127-4a19-b745-998955786847'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:38.460058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037ab3ea-1188-4e73-8f49-f83c6b034c98 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:38.467865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99865adc-2127-4a19-b745-998955786847 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae24b58d42174abeb14c231c5baada1c', 'request_id': 'req-99865adc-2127-4a19-b745-998955786847'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:38.476169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 944/1867] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:43:38.498249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99865adc-2127-4a19-b745-998955786847 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae24b58d42174abeb14c231c5baada1c', 'request_id': 'req-99865adc-2127-4a19-b745-998955786847'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:38.523841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99865adc-2127-4a19-b745-998955786847 admin admin] The token cache is being invalidate because project ae24b58d42174abeb14c231c5baada1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:38.524295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99865adc-2127-4a19-b745-998955786847 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:38.527647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99865adc-2127-4a19-b745-998955786847 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:38.534606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 924/1868] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:38 2022] DELETE /identity/v3/projects/ae24b58d42174abeb14c231c5baada1c => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:38.540165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c036547-8aaa-494b-8eda-084f917fedfb None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:38.540274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c036547-8aaa-494b-8eda-084f917fedfb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:38.540520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c036547-8aaa-494b-8eda-084f917fedfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:38.589899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-5c036547-8aaa-494b-8eda-084f917fedfb None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:38.595444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5c036547-8aaa-494b-8eda-084f917fedfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:38.750039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 945/1869] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:38.762872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578ce52-c468-48be-b356-aae299c9d141 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:38.762872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578ce52-c468-48be-b356-aae299c9d141 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:38.762872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5578ce52-c468-48be-b356-aae299c9d141 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:38.795084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5578ce52-c468-48be-b356-aae299c9d141 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:38.797135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5578ce52-c468-48be-b356-aae299c9d141 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:38.822039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 925/1870] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:38.823361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c036547-8aaa-494b-8eda-084f917fedfb None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:38.847527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c036547-8aaa-494b-8eda-084f917fedfb None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:38.868998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82196e3d-4df5-4b17-9bfb-9bf42f2dfeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:38.869521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82196e3d-4df5-4b17-9bfb-9bf42f2dfeab admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:38.869816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82196e3d-4df5-4b17-9bfb-9bf42f2dfeab admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:38.870095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82196e3d-4df5-4b17-9bfb-9bf42f2dfeab admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:38.870948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82196e3d-4df5-4b17-9bfb-9bf42f2dfeab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:38.871316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82196e3d-4df5-4b17-9bfb-9bf42f2dfeab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:38.877690 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82196e3d-4df5-4b17-9bfb-9bf42f2dfeab admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:38.889447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 946/1871] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:43:38.897096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578ce52-c468-48be-b356-aae299c9d141 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:38.907851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5578ce52-c468-48be-b356-aae299c9d141 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:38.912464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153169ed-7d5a-48f8-ab09-cdccfcb6261a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:38.913047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153169ed-7d5a-48f8-ab09-cdccfcb6261a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:38.913325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153169ed-7d5a-48f8-ab09-cdccfcb6261a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:38.913597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153169ed-7d5a-48f8-ab09-cdccfcb6261a admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:38.914271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153169ed-7d5a-48f8-ab09-cdccfcb6261a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:38.916574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153169ed-7d5a-48f8-ab09-cdccfcb6261a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:38.937368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82196e3d-4df5-4b17-9bfb-9bf42f2dfeab admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:38.941671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-82196e3d-4df5-4b17-9bfb-9bf42f2dfeab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:38.951588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 926/1872] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:38 2022] POST /identity/v3/projects => generated 391 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:38.956914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153169ed-7d5a-48f8-ab09-cdccfcb6261a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:38.970766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82196e3d-4df5-4b17-9bfb-9bf42f2dfeab admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:39.006477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:39.063838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153169ed-7d5a-48f8-ab09-cdccfcb6261a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:39.069373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be81b56-cac7-4d5b-830e-7ca59c145213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:39.070133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be81b56-cac7-4d5b-830e-7ca59c145213 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.070494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be81b56-cac7-4d5b-830e-7ca59c145213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.071274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be81b56-cac7-4d5b-830e-7ca59c145213 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.072191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be81b56-cac7-4d5b-830e-7ca59c145213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:39.074633 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be81b56-cac7-4d5b-830e-7ca59c145213 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:39.106019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 927/1873] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:38 2022] POST /identity/v3/users => generated 296 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:39.110450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be81b56-cac7-4d5b-830e-7ca59c145213 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:39.116691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:39.117400 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.118247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.118369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] PATH_INFO: `/v3/projects/6c94dcf829b74a6cb9c1c6ba2cbfeb75` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.119954 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c94dcf829b74a6cb9c1c6ba2cbfeb75)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:39.122029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:39.122196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be81b56-cac7-4d5b-830e-7ca59c145213 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:39.129417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8ebd9-96fb-4cd9-8719-4f7c5656ca4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:39.130035 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8ebd9-96fb-4cd9-8719-4f7c5656ca4f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.130334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8ebd9-96fb-4cd9-8719-4f7c5656ca4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.130569 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8ebd9-96fb-4cd9-8719-4f7c5656ca4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.131206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8ebd9-96fb-4cd9-8719-4f7c5656ca4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:39.133314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8ebd9-96fb-4cd9-8719-4f7c5656ca4f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:39.146291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 928/1874] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:39.149846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8ebd9-96fb-4cd9-8719-4f7c5656ca4f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:39.165843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8ebd9-96fb-4cd9-8719-4f7c5656ca4f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:39.170559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:39.172425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62253b9f-9c6b-484c-9429-e023a0349cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:39.173436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62253b9f-9c6b-484c-9429-e023a0349cf1 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.173858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62253b9f-9c6b-484c-9429-e023a0349cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.174219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62253b9f-9c6b-484c-9429-e023a0349cf1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.175113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62253b9f-9c6b-484c-9429-e023a0349cf1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:39.177344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62253b9f-9c6b-484c-9429-e023a0349cf1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:39.196803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 929/1875] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:39.201044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62253b9f-9c6b-484c-9429-e023a0349cf1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:39.213035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c94dcf829b74a6cb9c1c6ba2cbfeb75', 'request_id': 'req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:39.221284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62253b9f-9c6b-484c-9429-e023a0349cf1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:39.230816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c94dcf829b74a6cb9c1c6ba2cbfeb75', 'request_id': 'req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:39.248419 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c94dcf829b74a6cb9c1c6ba2cbfeb75', 'request_id': 'req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:39.267051 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] The token cache is being invalidate because project 6c94dcf829b74a6cb9c1c6ba2cbfeb75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:39.267274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:39.267966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:39.270592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 947/1876] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:38 2022] DELETE /identity/v3/projects/6c94dcf829b74a6cb9c1c6ba2cbfeb75 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:39.275193 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:39.278360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:39.326592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9f0c18-b9f9-404a-b029-5e39e0816846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:39.327545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f0c18-b9f9-404a-b029-5e39e0816846 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.327964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f0c18-b9f9-404a-b029-5e39e0816846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.329044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9f0c18-b9f9-404a-b029-5e39e0816846 admin admin] PATH_INFO: `/v3/projects/35ab8e4d17ba43c0ab828fcbd7593931/users/813dbc861cf64f59bcdf7ab81b683752/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.363107 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f0c18-b9f9-404a-b029-5e39e0816846 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35ab8e4d17ba43c0ab828fcbd7593931, user_id=813dbc861cf64f59bcdf7ab81b683752, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:39.368814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9f0c18-b9f9-404a-b029-5e39e0816846 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:39.387647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 930/1877] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:39 2022] PUT /identity/v3/projects/35ab8e4d17ba43c0ab828fcbd7593931/users/813dbc861cf64f59bcdf7ab81b683752/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:39.399443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576cb70-b481-46dd-a1ad-ed0f875baa60 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.399806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576cb70-b481-46dd-a1ad-ed0f875baa60 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.400081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1576cb70-b481-46dd-a1ad-ed0f875baa60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.410382 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4003e87c-92cd-4e54-9b68-6aebf5eff3d3 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:39.445600 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1576cb70-b481-46dd-a1ad-ed0f875baa60 None None] MFA Rules not processed for user `813dbc861cf64f59bcdf7ab81b683752`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:39.446834 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1576cb70-b481-46dd-a1ad-ed0f875baa60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:39.481073 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:39.482137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.482137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.482429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] PATH_INFO: `/v3/users/c06738a789ec46e69790e093678f380b` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.483651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c06738a789ec46e69790e093678f380b)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:39.486858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:39.520722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 931/1878] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:39 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:39.524886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576cb70-b481-46dd-a1ad-ed0f875baa60 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:39.526826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1576cb70-b481-46dd-a1ad-ed0f875baa60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:39.554406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1576cb70-b481-46dd-a1ad-ed0f875baa60 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:39.572387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bbc0c-12a5-4543-aa62-c8a27b8e07c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:39.573096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bbc0c-12a5-4543-aa62-c8a27b8e07c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.573297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bbc0c-12a5-4543-aa62-c8a27b8e07c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.573620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199bbc0c-12a5-4543-aa62-c8a27b8e07c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.574261 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bbc0c-12a5-4543-aa62-c8a27b8e07c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:39.575759 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c06738a789ec46e69790e093678f380b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:39.577385 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199bbc0c-12a5-4543-aa62-c8a27b8e07c1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:39.585444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:39.588220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c06738a789ec46e69790e093678f380b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:39.588918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c06738a789ec46e69790e093678f380b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:39.602590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:39.605703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c06738a789ec46e69790e093678f380b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:39.614402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 932/1879] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:39 2022] POST /identity/v3/projects => generated 391 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:39.616717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c06738a789ec46e69790e093678f380b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:39.618324 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bbc0c-12a5-4543-aa62-c8a27b8e07c1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:39.619969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-199bbc0c-12a5-4543-aa62-c8a27b8e07c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:39.628807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 948/1880] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:39 2022] DELETE /identity/v3/users/c06738a789ec46e69790e093678f380b => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:39.705195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199bbc0c-12a5-4543-aa62-c8a27b8e07c1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:39.780750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75516f-ab01-49e2-a32b-136f44a005d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:39.782072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75516f-ab01-49e2-a32b-136f44a005d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.782350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75516f-ab01-49e2-a32b-136f44a005d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.782723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a75516f-ab01-49e2-a32b-136f44a005d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.783383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75516f-ab01-49e2-a32b-136f44a005d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:39.787331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a75516f-ab01-49e2-a32b-136f44a005d3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:39.808606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 933/1881] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:39 2022] POST /identity/v3/users => generated 302 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:39.812845 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:39.825353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f50da14-d735-40bd-a140-e34c92c32fa4 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:39.831323 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca8c6-cf3c-4dc1-90aa-5b57e6551b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:39.831931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca8c6-cf3c-4dc1-90aa-5b57e6551b88 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.832203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca8c6-cf3c-4dc1-90aa-5b57e6551b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.832470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878ca8c6-cf3c-4dc1-90aa-5b57e6551b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.833144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ca8c6-cf3c-4dc1-90aa-5b57e6551b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:39.834753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878ca8c6-cf3c-4dc1-90aa-5b57e6551b88 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:39.845646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 949/1882] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:43:39.849719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75516f-ab01-49e2-a32b-136f44a005d3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:39.863832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a75516f-ab01-49e2-a32b-136f44a005d3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:39.869346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694406e-73d5-4aaa-9517-33ff61c1ceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:39.870248 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694406e-73d5-4aaa-9517-33ff61c1ceac admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.870561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694406e-73d5-4aaa-9517-33ff61c1ceac admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.872299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e694406e-73d5-4aaa-9517-33ff61c1ceac admin admin] PATH_INFO: `/v3/projects/a37f171018c04854bd99a88526d5b44f/users/adcf9cf7f6c742be876d183e7e17c7d5/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.891518 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694406e-73d5-4aaa-9517-33ff61c1ceac admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37f171018c04854bd99a88526d5b44f, user_id=adcf9cf7f6c742be876d183e7e17c7d5, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:39.895042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e694406e-73d5-4aaa-9517-33ff61c1ceac admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:39.909471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 934/1883] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:39 2022] PUT /identity/v3/projects/a37f171018c04854bd99a88526d5b44f/users/adcf9cf7f6c742be876d183e7e17c7d5/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:39.913599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca8c6-cf3c-4dc1-90aa-5b57e6551b88 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:39.923896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878ca8c6-cf3c-4dc1-90aa-5b57e6551b88 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:39.989036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b0e03-673b-41a7-ba2e-5b6979220116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:39.989732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0e03-673b-41a7-ba2e-5b6979220116 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:39.990219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0e03-673b-41a7-ba2e-5b6979220116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:39.990551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509b0e03-673b-41a7-ba2e-5b6979220116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:39.991263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b0e03-673b-41a7-ba2e-5b6979220116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:39.993008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509b0e03-673b-41a7-ba2e-5b6979220116 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:40.005392 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 950/1884] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:43:40.014056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694406e-73d5-4aaa-9517-33ff61c1ceac admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:40.016485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b0e03-673b-41a7-ba2e-5b6979220116 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:40.020534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-509b0e03-673b-41a7-ba2e-5b6979220116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:40.035233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e694406e-73d5-4aaa-9517-33ff61c1ceac admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:40.036017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509b0e03-673b-41a7-ba2e-5b6979220116 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:40.044594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:40.045649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.046106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.046582 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] PATH_INFO: `/v3/projects/93d751d7d1424c3a83c6c663d6dffc8f` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.048855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=93d751d7d1424c3a83c6c663d6dffc8f)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:40.052796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:40.054174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac87d2f-b8b2-4886-a068-b4474d2fffa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:40.055136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87d2f-b8b2-4886-a068-b4474d2fffa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.055555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87d2f-b8b2-4886-a068-b4474d2fffa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.055964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac87d2f-b8b2-4886-a068-b4474d2fffa8 admin admin] PATH_INFO: `/v3/projects/a37f171018c04854bd99a88526d5b44f/users/adcf9cf7f6c742be876d183e7e17c7d5/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.058222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac87d2f-b8b2-4886-a068-b4474d2fffa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a37f171018c04854bd99a88526d5b44f, user_id=adcf9cf7f6c742be876d183e7e17c7d5, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:40.061181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac87d2f-b8b2-4886-a068-b4474d2fffa8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:40.079500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 935/1885] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:40 2022] PUT /identity/v3/projects/a37f171018c04854bd99a88526d5b44f/users/adcf9cf7f6c742be876d183e7e17c7d5/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:40.096852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc7a0b-94c2-47b2-8888-83afaad54d1c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.097161 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc7a0b-94c2-47b2-8888-83afaad54d1c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.097415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38dc7a0b-94c2-47b2-8888-83afaad54d1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.124426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93d751d7d1424c3a83c6c663d6dffc8f', 'request_id': 'req-6156ac3a-9799-4a6b-87c5-860de0f041ab'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:40.127952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-38dc7a0b-94c2-47b2-8888-83afaad54d1c None None] MFA Rules not processed for user `adcf9cf7f6c742be876d183e7e17c7d5`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:40.129343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-38dc7a0b-94c2-47b2-8888-83afaad54d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:40.133690 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93d751d7d1424c3a83c6c663d6dffc8f', 'request_id': 'req-6156ac3a-9799-4a6b-87c5-860de0f041ab'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:40.139640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93d751d7d1424c3a83c6c663d6dffc8f', 'request_id': 'req-6156ac3a-9799-4a6b-87c5-860de0f041ab'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:40.153890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] The token cache is being invalidate because project 93d751d7d1424c3a83c6c663d6dffc8f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:40.154401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:40.155065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6156ac3a-9799-4a6b-87c5-860de0f041ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:40.157298 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 951/1886] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:40 2022] DELETE /identity/v3/projects/93d751d7d1424c3a83c6c663d6dffc8f => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:40.162527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3dc44-ed01-459e-8b4f-78381bfa97ba None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.162816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3dc44-ed01-459e-8b4f-78381bfa97ba None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.163087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e3dc44-ed01-459e-8b4f-78381bfa97ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.164730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:40.184305 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-a8e3dc44-ed01-459e-8b4f-78381bfa97ba None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:40.185562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a8e3dc44-ed01-459e-8b4f-78381bfa97ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:40.207479 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 936/1887] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:40.218229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc7a0b-94c2-47b2-8888-83afaad54d1c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:40.220204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-38dc7a0b-94c2-47b2-8888-83afaad54d1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:40.234135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:40.248386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 952/1888] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:40.258538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7309521-3a2b-428b-bf45-a8a8dc7448e1 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.259255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7309521-3a2b-428b-bf45-a8a8dc7448e1 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.259255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7309521-3a2b-428b-bf45-a8a8dc7448e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.283410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-a7309521-3a2b-428b-bf45-a8a8dc7448e1 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:40.284621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a7309521-3a2b-428b-bf45-a8a8dc7448e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:40.298791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 953/1889] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:40.301296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38dc7a0b-94c2-47b2-8888-83afaad54d1c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:40.302849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7309521-3a2b-428b-bf45-a8a8dc7448e1 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:40.305444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b106973-e8c1-4326-a635-c12c4cb8081e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:40.306062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b106973-e8c1-4326-a635-c12c4cb8081e service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.306342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b106973-e8c1-4326-a635-c12c4cb8081e service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.306616 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b106973-e8c1-4326-a635-c12c4cb8081e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.314369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7309521-3a2b-428b-bf45-a8a8dc7448e1 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:40.318964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b106973-e8c1-4326-a635-c12c4cb8081e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:40.319883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331082a-d862-443d-85fb-b42213fcbc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:40.320417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b106973-e8c1-4326-a635-c12c4cb8081e service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:40.320568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331082a-d862-443d-85fb-b42213fcbc82 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.320989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331082a-d862-443d-85fb-b42213fcbc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.321323 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3331082a-d862-443d-85fb-b42213fcbc82 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.322031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331082a-d862-443d-85fb-b42213fcbc82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:40.322395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331082a-d862-443d-85fb-b42213fcbc82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:40.324067 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3331082a-d862-443d-85fb-b42213fcbc82 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:40.331822 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 937/1890] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:40 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:43:40.341201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 954/1891] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:43:40.347505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b106973-e8c1-4326-a635-c12c4cb8081e service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:40.360295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b106973-e8c1-4326-a635-c12c4cb8081e service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:40.366454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04845aa-f8b8-4020-8b44-3e2c7d3bf72d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:40.367219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04845aa-f8b8-4020-8b44-3e2c7d3bf72d admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.367488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04845aa-f8b8-4020-8b44-3e2c7d3bf72d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.367802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04845aa-f8b8-4020-8b44-3e2c7d3bf72d admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.368792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04845aa-f8b8-4020-8b44-3e2c7d3bf72d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:40.371362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d04845aa-f8b8-4020-8b44-3e2c7d3bf72d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:40.412799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 938/1892] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:40 2022] POST /identity/v3/projects => generated 383 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:40.416859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331082a-d862-443d-85fb-b42213fcbc82 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:40.432850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3331082a-d862-443d-85fb-b42213fcbc82 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:40.440338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04845aa-f8b8-4020-8b44-3e2c7d3bf72d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:40.442171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d04845aa-f8b8-4020-8b44-3e2c7d3bf72d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:40.520700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c19ae8-3cb3-43c2-a0c2-03ca3e042a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:40.521772 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c19ae8-3cb3-43c2-a0c2-03ca3e042a17 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.522046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c19ae8-3cb3-43c2-a0c2-03ca3e042a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.522312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c19ae8-3cb3-43c2-a0c2-03ca3e042a17 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.522945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c19ae8-3cb3-43c2-a0c2-03ca3e042a17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:40.525034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c19ae8-3cb3-43c2-a0c2-03ca3e042a17 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:40.527131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d04845aa-f8b8-4020-8b44-3e2c7d3bf72d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:40.532316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd09f7-499b-4bd6-adb9-d26e2e1dfee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:40.533300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd09f7-499b-4bd6-adb9-d26e2e1dfee0 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.533810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd09f7-499b-4bd6-adb9-d26e2e1dfee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.534273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd09f7-499b-4bd6-adb9-d26e2e1dfee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.536331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-28fd09f7-499b-4bd6-adb9-d26e2e1dfee0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:40.577698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 955/1893] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:40 2022] POST /identity/v3/users => generated 292 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:40.598838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c19ae8-3cb3-43c2-a0c2-03ca3e042a17 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:40.615079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c19ae8-3cb3-43c2-a0c2-03ca3e042a17 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:40.627046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc52b8-3522-4dc3-952a-73875859d1e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:40.627678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc52b8-3522-4dc3-952a-73875859d1e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.627960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc52b8-3522-4dc3-952a-73875859d1e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.628234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dc52b8-3522-4dc3-952a-73875859d1e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.629487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc52b8-3522-4dc3-952a-73875859d1e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:40.632402 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dc52b8-3522-4dc3-952a-73875859d1e8 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:40.644988 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd09f7-499b-4bd6-adb9-d26e2e1dfee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:40.645487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd09f7-499b-4bd6-adb9-d26e2e1dfee0 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:40.663550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 939/1894] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:40 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:43:40.667869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 956/1895] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:43:40.673940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd09f7-499b-4bd6-adb9-d26e2e1dfee0 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:40.688293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:40.694537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd09f7-499b-4bd6-adb9-d26e2e1dfee0 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:40.702401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba73214-7e6c-41ce-acc1-b4ca944db46a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:40.704388 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73214-7e6c-41ce-acc1-b4ca944db46a admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.704960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73214-7e6c-41ce-acc1-b4ca944db46a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.705416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba73214-7e6c-41ce-acc1-b4ca944db46a admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.706379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba73214-7e6c-41ce-acc1-b4ca944db46a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:40.708296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba73214-7e6c-41ce-acc1-b4ca944db46a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:40.724868 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 940/1896] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:40 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:40.729393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc52b8-3522-4dc3-952a-73875859d1e8 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:40.743502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dc52b8-3522-4dc3-952a-73875859d1e8 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:40.749138 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f73c59-9938-4339-9841-a5e8461f9cda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:40.749979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f73c59-9938-4339-9841-a5e8461f9cda admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.750676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f73c59-9938-4339-9841-a5e8461f9cda admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.750960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f73c59-9938-4339-9841-a5e8461f9cda admin admin] PATH_INFO: `/v3/projects/df6b661b722f4a5da07ff036bab7acf1/users/2a2c2338316749829bff5e54f76cfcf3/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.773835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f73c59-9938-4339-9841-a5e8461f9cda admin admin] RBAC: Authorizing `identity:create_grant(project_id=df6b661b722f4a5da07ff036bab7acf1, user_id=2a2c2338316749829bff5e54f76cfcf3, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:40.776666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f73c59-9938-4339-9841-a5e8461f9cda admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:40.803822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 957/1897] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:40 2022] PUT /identity/v3/projects/df6b661b722f4a5da07ff036bab7acf1/users/2a2c2338316749829bff5e54f76cfcf3/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:40.817833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88caba2-8dd7-45ed-b0cb-ba60b855ed1f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:40.818159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88caba2-8dd7-45ed-b0cb-ba60b855ed1f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:40.818430 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88caba2-8dd7-45ed-b0cb-ba60b855ed1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:40.881513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c88caba2-8dd7-45ed-b0cb-ba60b855ed1f None None] MFA Rules not processed for user `2a2c2338316749829bff5e54f76cfcf3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:40.883074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c88caba2-8dd7-45ed-b0cb-ba60b855ed1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:41.007777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f73c59-9938-4339-9841-a5e8461f9cda admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:41.010256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 941/1898] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:41.137435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f73c59-9938-4339-9841-a5e8461f9cda admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:41.152273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afa4b3-39c4-4ca0-b036-152cfb9d8001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:41.153255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afa4b3-39c4-4ca0-b036-152cfb9d8001 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:41.153768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afa4b3-39c4-4ca0-b036-152cfb9d8001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:41.154151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7afa4b3-39c4-4ca0-b036-152cfb9d8001 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:41.155273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afa4b3-39c4-4ca0-b036-152cfb9d8001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:41.158224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7afa4b3-39c4-4ca0-b036-152cfb9d8001 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:41.218652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 958/1899] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:41 2022] POST /identity/v3/projects => generated 381 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:41.223717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88caba2-8dd7-45ed-b0cb-ba60b855ed1f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:41.240191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88caba2-8dd7-45ed-b0cb-ba60b855ed1f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:41.323407 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23525a8-c8a0-440f-973a-a1912c61130a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:41.324826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23525a8-c8a0-440f-973a-a1912c61130a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:41.325863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23525a8-c8a0-440f-973a-a1912c61130a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:41.325863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23525a8-c8a0-440f-973a-a1912c61130a admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:41.326519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23525a8-c8a0-440f-973a-a1912c61130a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:41.329510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23525a8-c8a0-440f-973a-a1912c61130a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:41.353308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 942/1900] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:41 2022] POST /identity/v3/users => generated 297 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:41.357347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afa4b3-39c4-4ca0-b036-152cfb9d8001 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:41.369913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7afa4b3-39c4-4ca0-b036-152cfb9d8001 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:41.376599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c5b2-56e8-4de1-aaa0-7b11b280acb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:41.377351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c5b2-56e8-4de1-aaa0-7b11b280acb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:41.377723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c5b2-56e8-4de1-aaa0-7b11b280acb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:41.378068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf2c5b2-56e8-4de1-aaa0-7b11b280acb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:41.378741 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2c5b2-56e8-4de1-aaa0-7b11b280acb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:41.380537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf2c5b2-56e8-4de1-aaa0-7b11b280acb5 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:41.393527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 959/1901] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:43:41.398028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23525a8-c8a0-440f-973a-a1912c61130a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:41.409121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23525a8-c8a0-440f-973a-a1912c61130a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:41.414141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9477b-8bcd-4216-8356-49c3b99b6d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:41.414803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9477b-8bcd-4216-8356-49c3b99b6d78 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:41.415102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9477b-8bcd-4216-8356-49c3b99b6d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:41.415370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb9477b-8bcd-4216-8356-49c3b99b6d78 admin admin] PATH_INFO: `/v3/projects/9052f5e67af04d4bab78281560d904e0/users/3c9cbfeb31c3435a9302e2b8169a1844/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:41.431628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9477b-8bcd-4216-8356-49c3b99b6d78 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9052f5e67af04d4bab78281560d904e0, user_id=3c9cbfeb31c3435a9302e2b8169a1844, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:41.445997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb9477b-8bcd-4216-8356-49c3b99b6d78 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:41.480815 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 943/1902] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:41 2022] PUT /identity/v3/projects/9052f5e67af04d4bab78281560d904e0/users/3c9cbfeb31c3435a9302e2b8169a1844/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:41.486102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c5b2-56e8-4de1-aaa0-7b11b280acb5 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:41.505027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:41.505027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf2c5b2-56e8-4de1-aaa0-7b11b280acb5 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:41.588379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699d66e-e1ee-4747-8914-06628215e71d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:41.589024 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699d66e-e1ee-4747-8914-06628215e71d admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:41.589297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699d66e-e1ee-4747-8914-06628215e71d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:41.589564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699d66e-e1ee-4747-8914-06628215e71d admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:41.590199 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699d66e-e1ee-4747-8914-06628215e71d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:41.591776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5699d66e-e1ee-4747-8914-06628215e71d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:41.602909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 960/1903] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:41 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:41.607197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9477b-8bcd-4216-8356-49c3b99b6d78 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:41.618582 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb9477b-8bcd-4216-8356-49c3b99b6d78 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:41.624844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd668311-7ec1-4f44-8d30-bdc6b68e4eeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:41.626477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668311-7ec1-4f44-8d30-bdc6b68e4eeb admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:41.626845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668311-7ec1-4f44-8d30-bdc6b68e4eeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:41.627198 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd668311-7ec1-4f44-8d30-bdc6b68e4eeb admin admin] PATH_INFO: `/v3/projects/9052f5e67af04d4bab78281560d904e0/users/3c9cbfeb31c3435a9302e2b8169a1844/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:41.629427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd668311-7ec1-4f44-8d30-bdc6b68e4eeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9052f5e67af04d4bab78281560d904e0, user_id=3c9cbfeb31c3435a9302e2b8169a1844, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:41.632153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd668311-7ec1-4f44-8d30-bdc6b68e4eeb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:41.644583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 944/1904] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:41 2022] PUT /identity/v3/projects/9052f5e67af04d4bab78281560d904e0/users/3c9cbfeb31c3435a9302e2b8169a1844/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:41.654504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55605117-ce00-4908-8fea-15f285d58be5 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:41.654831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55605117-ce00-4908-8fea-15f285d58be5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:41.655106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55605117-ce00-4908-8fea-15f285d58be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:41.690241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-55605117-ce00-4908-8fea-15f285d58be5 None None] MFA Rules not processed for user `3c9cbfeb31c3435a9302e2b8169a1844`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:41.691464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-55605117-ce00-4908-8fea-15f285d58be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:41.760534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 961/1905] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:41 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:41.774211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd668311-7ec1-4f44-8d30-bdc6b68e4eeb admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:41.815127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:41.832336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd668311-7ec1-4f44-8d30-bdc6b68e4eeb admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:41.835838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf08d1-756e-4037-a3f1-6eda07eba4bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:41.836431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf08d1-756e-4037-a3f1-6eda07eba4bf service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:41.836704 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf08d1-756e-4037-a3f1-6eda07eba4bf service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:41.837000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf08d1-756e-4037-a3f1-6eda07eba4bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:41.846522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf08d1-756e-4037-a3f1-6eda07eba4bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:41.847931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf08d1-756e-4037-a3f1-6eda07eba4bf service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:41.857402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 945/1906] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:41 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:41.954822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55605117-ce00-4908-8fea-15f285d58be5 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:41.964578 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55605117-ce00-4908-8fea-15f285d58be5 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:41.968337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f6b59-cea9-412c-b24e-9b99c115d878 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:41.969974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f6b59-cea9-412c-b24e-9b99c115d878 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:41.970092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f6b59-cea9-412c-b24e-9b99c115d878 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:41.970383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5f6b59-cea9-412c-b24e-9b99c115d878 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:42.037491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f6b59-cea9-412c-b24e-9b99c115d878 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:42.038915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5f6b59-cea9-412c-b24e-9b99c115d878 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:42.048697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 962/1907] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:42.320888 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf08d1-756e-4037-a3f1-6eda07eba4bf service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:42.324292 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7acf08d1-756e-4037-a3f1-6eda07eba4bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:42.451860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf08d1-756e-4037-a3f1-6eda07eba4bf service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:42.460569 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ef666-300d-47e4-8f9a-157ac56025ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:42.465484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ef666-300d-47e4-8f9a-157ac56025ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:42.465609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f6b59-cea9-412c-b24e-9b99c115d878 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:42.465876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ef666-300d-47e4-8f9a-157ac56025ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:42.466073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ef666-300d-47e4-8f9a-157ac56025ea admin admin] PATH_INFO: `/v3/users/cd053e1b142b438cb09435855a7a6b65` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:42.467860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ef666-300d-47e4-8f9a-157ac56025ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd053e1b142b438cb09435855a7a6b65)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:42.470109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ef666-300d-47e4-8f9a-157ac56025ea admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:42.488795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5f6b59-cea9-412c-b24e-9b99c115d878 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:42.498212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b10dc32-8927-4056-a883-4456a942b7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:42.498993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10dc32-8927-4056-a883-4456a942b7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:42.500251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10dc32-8927-4056-a883-4456a942b7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:42.501149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b10dc32-8927-4056-a883-4456a942b7c9 admin admin] PATH_INFO: `/v3/users/2a2c2338316749829bff5e54f76cfcf3` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:42.503685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10dc32-8927-4056-a883-4456a942b7c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a2c2338316749829bff5e54f76cfcf3)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:42.508483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b10dc32-8927-4056-a883-4456a942b7c9 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:42.550010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:42.619522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-328ef666-300d-47e4-8f9a-157ac56025ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd053e1b142b438cb09435855a7a6b65'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:42.620316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-328ef666-300d-47e4-8f9a-157ac56025ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd053e1b142b438cb09435855a7a6b65'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:42.634928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-328ef666-300d-47e4-8f9a-157ac56025ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd053e1b142b438cb09435855a7a6b65'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:42.635950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-1b10dc32-8927-4056-a883-4456a942b7c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a2c2338316749829bff5e54f76cfcf3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:42.649364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-328ef666-300d-47e4-8f9a-157ac56025ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd053e1b142b438cb09435855a7a6b65'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:42.657908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-1b10dc32-8927-4056-a883-4456a942b7c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a2c2338316749829bff5e54f76cfcf3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:42.658728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-1b10dc32-8927-4056-a883-4456a942b7c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a2c2338316749829bff5e54f76cfcf3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:42.677585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-1b10dc32-8927-4056-a883-4456a942b7c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a2c2338316749829bff5e54f76cfcf3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:42.681330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-328ef666-300d-47e4-8f9a-157ac56025ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd053e1b142b438cb09435855a7a6b65'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:42.693948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-1b10dc32-8927-4056-a883-4456a942b7c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a2c2338316749829bff5e54f76cfcf3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:42.694697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 946/1908] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:42 2022] DELETE /identity/v3/users/cd053e1b142b438cb09435855a7a6b65 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:42.699073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a79416-e0e2-417d-beb9-bd6d84c58772 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:42.699463 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a79416-e0e2-417d-beb9-bd6d84c58772 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:42.699839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a79416-e0e2-417d-beb9-bd6d84c58772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:42.713691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 963/1909] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:42 2022] DELETE /identity/v3/users/2a2c2338316749829bff5e54f76cfcf3 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:42.715566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec21b8-aee7-41ae-85fb-74ac00241ac2 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:42.715920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec21b8-aee7-41ae-85fb-74ac00241ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:42.716250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beec21b8-aee7-41ae-85fb-74ac00241ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:42.756814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a9a79416-e0e2-417d-beb9-bd6d84c58772 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:42.758836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a9a79416-e0e2-417d-beb9-bd6d84c58772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:42.767007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-beec21b8-aee7-41ae-85fb-74ac00241ac2 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:42.770342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-beec21b8-aee7-41ae-85fb-74ac00241ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:42.899739 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 964/1910] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:42.930264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec21b8-aee7-41ae-85fb-74ac00241ac2 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:42.931650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:42.931832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-beec21b8-aee7-41ae-85fb-74ac00241ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:42.972059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 947/1911] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:42.976459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a79416-e0e2-417d-beb9-bd6d84c58772 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:42.979345 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a9a79416-e0e2-417d-beb9-bd6d84c58772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:43.065343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beec21b8-aee7-41ae-85fb-74ac00241ac2 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:43.073488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c8405-ed62-4a69-b8b6-88e221c77060 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:43.074207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c8405-ed62-4a69-b8b6-88e221c77060 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:43.074824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c8405-ed62-4a69-b8b6-88e221c77060 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:43.075114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c8405-ed62-4a69-b8b6-88e221c77060 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:43.093096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c8405-ed62-4a69-b8b6-88e221c77060 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:43.097192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c8405-ed62-4a69-b8b6-88e221c77060 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:43.132159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a79416-e0e2-417d-beb9-bd6d84c58772 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:43.136987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ffe55-0813-4b9e-af6d-bfaabf505389 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:43.137513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ffe55-0813-4b9e-af6d-bfaabf505389 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:43.137792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ffe55-0813-4b9e-af6d-bfaabf505389 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:43.139312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1ffe55-0813-4b9e-af6d-bfaabf505389 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:43.141859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 965/1912] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:43:43.162675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ffe55-0813-4b9e-af6d-bfaabf505389 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:43.164306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc1ffe55-0813-4b9e-af6d-bfaabf505389 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:43.186548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 948/1913] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:43:44.221948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c8405-ed62-4a69-b8b6-88e221c77060 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:44.225488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-836c8405-ed62-4a69-b8b6-88e221c77060 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:44.239269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ffe55-0813-4b9e-af6d-bfaabf505389 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:44.240852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cc1ffe55-0813-4b9e-af6d-bfaabf505389 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:44.255974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c8405-ed62-4a69-b8b6-88e221c77060 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:44.265355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:44.267384 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:44.267833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:44.268253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] PATH_INFO: `/v3/projects/df6b661b722f4a5da07ff036bab7acf1` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:44.270546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] RBAC: Authorizing `identity:delete_project(project_id=df6b661b722f4a5da07ff036bab7acf1)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:44.273028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:44.315992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1ffe55-0813-4b9e-af6d-bfaabf505389 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:44.330531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:44.332238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:44.332747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:44.333428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] PATH_INFO: `/v3/projects/f13784b3824c495193bbf96fb6a2246d` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:44.335052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=f13784b3824c495193bbf96fb6a2246d)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:44.347606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:44.350130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:44.367913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df6b661b722f4a5da07ff036bab7acf1', 'request_id': 'req-70046ce5-cdc1-4a6c-a850-b91791e58dbb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.379478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df6b661b722f4a5da07ff036bab7acf1', 'request_id': 'req-70046ce5-cdc1-4a6c-a850-b91791e58dbb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.386816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df6b661b722f4a5da07ff036bab7acf1', 'request_id': 'req-70046ce5-cdc1-4a6c-a850-b91791e58dbb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.401508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] The token cache is being invalidate because project df6b661b722f4a5da07ff036bab7acf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:44.401742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f13784b3824c495193bbf96fb6a2246d', 'request_id': 'req-4df6b431-3092-455f-9204-d0c4d9d0de7a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.401833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.402651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.404355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 966/1914] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:44 2022] DELETE /identity/v3/projects/df6b661b722f4a5da07ff036bab7acf1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:44.408677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:44.409576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f13784b3824c495193bbf96fb6a2246d', 'request_id': 'req-4df6b431-3092-455f-9204-d0c4d9d0de7a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.410149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:44.418939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f13784b3824c495193bbf96fb6a2246d', 'request_id': 'req-4df6b431-3092-455f-9204-d0c4d9d0de7a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.432117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] The token cache is being invalidate because project f13784b3824c495193bbf96fb6a2246d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:44.432403 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.433081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.434809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 949/1915] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:44 2022] DELETE /identity/v3/projects/f13784b3824c495193bbf96fb6a2246d => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:44.439202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:44.440724 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:44.495504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70046ce5-cdc1-4a6c-a850-b91791e58dbb admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:44.517294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df6b431-3092-455f-9204-d0c4d9d0de7a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:44.522929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:44.523946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:44.524509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:44.525025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] PATH_INFO: `/v3/users/56b54f6d03e948599d2c6406e333b676` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:44.526788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] RBAC: Authorizing `identity:delete_user(user_id=56b54f6d03e948599d2c6406e333b676)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:44.530164 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:44.579357 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b03cc-2f16-467c-abb7-a55c38f47f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:44.580590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b03cc-2f16-467c-abb7-a55c38f47f41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:44.581135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b03cc-2f16-467c-abb7-a55c38f47f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:44.581644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b03cc-2f16-467c-abb7-a55c38f47f41 admin admin] PATH_INFO: `/v3/users/3c9cbfeb31c3435a9302e2b8169a1844` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:44.582956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b03cc-2f16-467c-abb7-a55c38f47f41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c9cbfeb31c3435a9302e2b8169a1844)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:44.583488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:44.585618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b03cc-2f16-467c-abb7-a55c38f47f41 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:44.612382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56b54f6d03e948599d2c6406e333b676'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.613679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56b54f6d03e948599d2c6406e333b676'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.623111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56b54f6d03e948599d2c6406e333b676'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.633334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56b54f6d03e948599d2c6406e333b676'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.647509 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-453b03cc-2f16-467c-abb7-a55c38f47f41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c9cbfeb31c3435a9302e2b8169a1844'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.649965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56b54f6d03e948599d2c6406e333b676'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.662629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-453b03cc-2f16-467c-abb7-a55c38f47f41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c9cbfeb31c3435a9302e2b8169a1844'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.662885 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 950/1916] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:44 2022] DELETE /identity/v3/users/56b54f6d03e948599d2c6406e333b676 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:44.663634 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-453b03cc-2f16-467c-abb7-a55c38f47f41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c9cbfeb31c3435a9302e2b8169a1844'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.702004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:44.709619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-453b03cc-2f16-467c-abb7-a55c38f47f41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c9cbfeb31c3435a9302e2b8169a1844'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.728934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-453b03cc-2f16-467c-abb7-a55c38f47f41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c9cbfeb31c3435a9302e2b8169a1844'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:44.756179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 967/1917] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:44 2022] DELETE /identity/v3/users/3c9cbfeb31c3435a9302e2b8169a1844 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:45.061584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:45.064481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:45.148358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53a27d2-eb66-4b1c-adee-289e5d38ccca admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:45.221536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:45.222195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:45.222473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:45.222741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] PATH_INFO: `/v3/users/813dbc861cf64f59bcdf7ab81b683752` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:45.224560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=813dbc861cf64f59bcdf7ab81b683752)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:45.227998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:45.354144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '813dbc861cf64f59bcdf7ab81b683752'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:45.361279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '813dbc861cf64f59bcdf7ab81b683752'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:45.370859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:45.385515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '813dbc861cf64f59bcdf7ab81b683752'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:45.405455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '813dbc861cf64f59bcdf7ab81b683752'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:45.436530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '813dbc861cf64f59bcdf7ab81b683752'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:45.446782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 951/1918] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:45 2022] DELETE /identity/v3/users/813dbc861cf64f59bcdf7ab81b683752 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:45.451627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae249c33-f1cf-40c3-9b74-42b6a2557b36 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:45.452136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae249c33-f1cf-40c3-9b74-42b6a2557b36 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:45.452336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae249c33-f1cf-40c3-9b74-42b6a2557b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:45.512132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-ae249c33-f1cf-40c3-9b74-42b6a2557b36 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:45.515160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ae249c33-f1cf-40c3-9b74-42b6a2557b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:45.584900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:45.589484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:45.715980 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 968/1919] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:45.730340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae249c33-f1cf-40c3-9b74-42b6a2557b36 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:45.733516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ae249c33-f1cf-40c3-9b74-42b6a2557b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:45.734416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d935cf-305a-4d2b-9fe4-d9c8c073bdf3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:45.739622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:45.740518 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:45.740946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:45.741233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] PATH_INFO: `/v3/projects/7d1a428e155e43119408c757abfe1019` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:45.742406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d1a428e155e43119408c757abfe1019)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:45.744325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:45.895137 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d1a428e155e43119408c757abfe1019', 'request_id': 'req-6f64c380-df64-4d89-849f-38cd53dc4b68'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:45.910837 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d1a428e155e43119408c757abfe1019', 'request_id': 'req-6f64c380-df64-4d89-849f-38cd53dc4b68'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:45.918629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae249c33-f1cf-40c3-9b74-42b6a2557b36 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:45.930476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d1a428e155e43119408c757abfe1019', 'request_id': 'req-6f64c380-df64-4d89-849f-38cd53dc4b68'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:45.954855 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] The token cache is being invalidate because project 7d1a428e155e43119408c757abfe1019 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:45.955165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:45.956016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:45.961074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 952/1920] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:45 2022] DELETE /identity/v3/projects/7d1a428e155e43119408c757abfe1019 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:45.961074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:45.962083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:46.003214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc262110-aff4-472d-ac08-acbfbe5b2d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.013614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc262110-aff4-472d-ac08-acbfbe5b2d6b service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.014989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc262110-aff4-472d-ac08-acbfbe5b2d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.015312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc262110-aff4-472d-ac08-acbfbe5b2d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.018162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-cc262110-aff4-472d-ac08-acbfbe5b2d6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:46.108232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f64c380-df64-4d89-849f-38cd53dc4b68 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:46.120129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.121985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.122797 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.123089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] PATH_INFO: `/v3/projects/9052f5e67af04d4bab78281560d904e0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.131265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=9052f5e67af04d4bab78281560d904e0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.131471 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:46.133033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc262110-aff4-472d-ac08-acbfbe5b2d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.134498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc262110-aff4-472d-ac08-acbfbe5b2d6b service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.136146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.142173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:46.151262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 969/1921] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 423 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:46.152676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c293-654f-4040-8344-a5be18679afd None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.153005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c293-654f-4040-8344-a5be18679afd None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.153283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a4c293-654f-4040-8344-a5be18679afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.194051 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-82a4c293-654f-4040-8344-a5be18679afd None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:46.195796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-82a4c293-654f-4040-8344-a5be18679afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:46.212670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 970/1922] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:46.215313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f54aa3-f671-4c6d-9854-e9d01196fb81 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.215687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f54aa3-f671-4c6d-9854-e9d01196fb81 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.215966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f54aa3-f671-4c6d-9854-e9d01196fb81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.227233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9052f5e67af04d4bab78281560d904e0', 'request_id': 'req-885267c7-ca33-421d-9bdf-131db31ed2ba'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:46.240980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9052f5e67af04d4bab78281560d904e0', 'request_id': 'req-885267c7-ca33-421d-9bdf-131db31ed2ba'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:46.244405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-08f54aa3-f671-4c6d-9854-e9d01196fb81 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:46.246371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-08f54aa3-f671-4c6d-9854-e9d01196fb81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:46.260864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9052f5e67af04d4bab78281560d904e0', 'request_id': 'req-885267c7-ca33-421d-9bdf-131db31ed2ba'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:46.278145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] The token cache is being invalidate because project 9052f5e67af04d4bab78281560d904e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:46.278391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:46.279104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-885267c7-ca33-421d-9bdf-131db31ed2ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:46.280674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 953/1923] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:45 2022] DELETE /identity/v3/projects/9052f5e67af04d4bab78281560d904e0 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:46.287259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846670a-360a-4bea-a2ad-e9cfa4e8505d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.287623 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846670a-360a-4bea-a2ad-e9cfa4e8505d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.288010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4846670a-360a-4bea-a2ad-e9cfa4e8505d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.310311 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4846670a-360a-4bea-a2ad-e9cfa4e8505d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:46.311754 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4846670a-360a-4bea-a2ad-e9cfa4e8505d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:46.371318 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 971/1924] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:46.374896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f54aa3-f671-4c6d-9854-e9d01196fb81 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:46.413362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f54aa3-f671-4c6d-9854-e9d01196fb81 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:46.424254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35f233-49c4-41e7-90e9-b711ce0203fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.424843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 954/1925] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:46.425273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35f233-49c4-41e7-90e9-b711ce0203fb admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.425273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35f233-49c4-41e7-90e9-b711ce0203fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.425483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a35f233-49c4-41e7-90e9-b711ce0203fb admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.426127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35f233-49c4-41e7-90e9-b711ce0203fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:46.426425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35f233-49c4-41e7-90e9-b711ce0203fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.428802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a35f233-49c4-41e7-90e9-b711ce0203fb admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.431324 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb26401-6969-4eb4-b25b-52c7e6b41084 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.431903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb26401-6969-4eb4-b25b-52c7e6b41084 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.433268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb26401-6969-4eb4-b25b-52c7e6b41084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.442937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 972/1926] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:43:46.447033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35f233-49c4-41e7-90e9-b711ce0203fb admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:46.461232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a35f233-49c4-41e7-90e9-b711ce0203fb admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:46.467920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2bb26401-6969-4eb4-b25b-52c7e6b41084 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:46.469531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2bb26401-6969-4eb4-b25b-52c7e6b41084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:46.475008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33968c9b-8f6e-4d34-8622-5b1dcc4dbc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.475626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33968c9b-8f6e-4d34-8622-5b1dcc4dbc16 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.475906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33968c9b-8f6e-4d34-8622-5b1dcc4dbc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.476178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33968c9b-8f6e-4d34-8622-5b1dcc4dbc16 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.476837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33968c9b-8f6e-4d34-8622-5b1dcc4dbc16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.478737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33968c9b-8f6e-4d34-8622-5b1dcc4dbc16 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.489897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 955/1927] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:46.494425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb26401-6969-4eb4-b25b-52c7e6b41084 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:46.506571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb26401-6969-4eb4-b25b-52c7e6b41084 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:46.511755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02207357-4a00-4f8c-957c-cc5af508e6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.512368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207357-4a00-4f8c-957c-cc5af508e6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.512641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207357-4a00-4f8c-957c-cc5af508e6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.512945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02207357-4a00-4f8c-957c-cc5af508e6b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.513795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02207357-4a00-4f8c-957c-cc5af508e6b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:46.514100 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02207357-4a00-4f8c-957c-cc5af508e6b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.516593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02207357-4a00-4f8c-957c-cc5af508e6b1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.518222 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 973/1928] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:46 2022] POST /identity/v3/projects => generated 389 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:46.522564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33968c9b-8f6e-4d34-8622-5b1dcc4dbc16 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:46.534637 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 956/1929] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:43:46.539088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02207357-4a00-4f8c-957c-cc5af508e6b1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:46.552550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02207357-4a00-4f8c-957c-cc5af508e6b1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:46.634950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0533ce0-2540-4505-9778-1063ecbd725c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.635677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533ce0-2540-4505-9778-1063ecbd725c admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.636032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533ce0-2540-4505-9778-1063ecbd725c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.636379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533ce0-2540-4505-9778-1063ecbd725c admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.637277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0533ce0-2540-4505-9778-1063ecbd725c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.639307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0533ce0-2540-4505-9778-1063ecbd725c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.658146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:46.677612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33968c9b-8f6e-4d34-8622-5b1dcc4dbc16 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:46.680748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 957/1930] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:46 2022] POST /identity/v3/projects => generated 385 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:46.681687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0533ce0-2540-4505-9778-1063ecbd725c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:46.779737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0533ce0-2540-4505-9778-1063ecbd725c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:46.790691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080174b8-0a13-483c-b937-3ff252305369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.790691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080174b8-0a13-483c-b937-3ff252305369 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.790691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080174b8-0a13-483c-b937-3ff252305369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.790691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080174b8-0a13-483c-b937-3ff252305369 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.791275 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080174b8-0a13-483c-b937-3ff252305369 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.791368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d28b1-6c75-4684-a8a5-e2ea79bf64b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.791368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d28b1-6c75-4684-a8a5-e2ea79bf64b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.791368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d28b1-6c75-4684-a8a5-e2ea79bf64b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.791652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080174b8-0a13-483c-b937-3ff252305369 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.791744 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43d28b1-6c75-4684-a8a5-e2ea79bf64b9 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.791744 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d28b1-6c75-4684-a8a5-e2ea79bf64b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.793494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43d28b1-6c75-4684-a8a5-e2ea79bf64b9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.799915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:46.823569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 974/1931] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:46 2022] POST /identity/v3/users => generated 295 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:46.828147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 958/1932] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:46 2022] POST /identity/v3/users => generated 293 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:46.829842 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080174b8-0a13-483c-b937-3ff252305369 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:46.832103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d28b1-6c75-4684-a8a5-e2ea79bf64b9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:46.843476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080174b8-0a13-483c-b937-3ff252305369 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:46.850130 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43d28b1-6c75-4684-a8a5-e2ea79bf64b9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:46.850577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bca24d-7b12-4220-95cd-032dccacf8a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.851440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bca24d-7b12-4220-95cd-032dccacf8a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.851881 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bca24d-7b12-4220-95cd-032dccacf8a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.852288 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bca24d-7b12-4220-95cd-032dccacf8a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.853137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bca24d-7b12-4220-95cd-032dccacf8a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.855378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bca24d-7b12-4220-95cd-032dccacf8a8 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.870620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165e59e-5eb4-49e2-9227-dd8b3b39ce33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.872327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165e59e-5eb4-49e2-9227-dd8b3b39ce33 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.872568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165e59e-5eb4-49e2-9227-dd8b3b39ce33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.876933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4165e59e-5eb4-49e2-9227-dd8b3b39ce33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.879488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165e59e-5eb4-49e2-9227-dd8b3b39ce33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.883595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4165e59e-5eb4-49e2-9227-dd8b3b39ce33 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.885450 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 975/1933] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:46.903395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bca24d-7b12-4220-95cd-032dccacf8a8 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:46.912913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 959/1934] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:46.915124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165e59e-5eb4-49e2-9227-dd8b3b39ce33 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:46.933962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bca24d-7b12-4220-95cd-032dccacf8a8 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:46.941178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4165e59e-5eb4-49e2-9227-dd8b3b39ce33 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:46.948713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345a37e-a4f1-4eef-867e-c5da636e5455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.949873 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d53fe7-91da-40eb-b720-7af8ffe7ef15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:46.950597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d53fe7-91da-40eb-b720-7af8ffe7ef15 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.950974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345a37e-a4f1-4eef-867e-c5da636e5455 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:46.951115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d53fe7-91da-40eb-b720-7af8ffe7ef15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.951771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d53fe7-91da-40eb-b720-7af8ffe7ef15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.952252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d53fe7-91da-40eb-b720-7af8ffe7ef15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.953578 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345a37e-a4f1-4eef-867e-c5da636e5455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:46.954047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2345a37e-a4f1-4eef-867e-c5da636e5455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:46.955285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345a37e-a4f1-4eef-867e-c5da636e5455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:46.966571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d53fe7-91da-40eb-b720-7af8ffe7ef15 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.970956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2345a37e-a4f1-4eef-867e-c5da636e5455 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:46.996708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 960/1935] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:46.998185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d53fe7-91da-40eb-b720-7af8ffe7ef15 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:46.998836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 976/1936] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:46 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:47.003087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345a37e-a4f1-4eef-867e-c5da636e5455 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:47.027376 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d53fe7-91da-40eb-b720-7af8ffe7ef15 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:47.030549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2345a37e-a4f1-4eef-867e-c5da636e5455 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:47.048386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b0af98-354c-45d9-acda-d583fb3b9fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:47.049312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0af98-354c-45d9-acda-d583fb3b9fbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:47.050208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0af98-354c-45d9-acda-d583fb3b9fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:47.050208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b0af98-354c-45d9-acda-d583fb3b9fbd admin admin] PATH_INFO: `/v3/projects/cc42dc9a609d441e848320a89013ad26/users/e0b603469c0f4d4fa5a332aec545aca9/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:47.050864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207b5e3-5211-4193-85ab-335783e2cd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:47.051539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b5e3-5211-4193-85ab-335783e2cd49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:47.051818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b5e3-5211-4193-85ab-335783e2cd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:47.052107 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5207b5e3-5211-4193-85ab-335783e2cd49 admin admin] PATH_INFO: `/v3/projects/6268f144e4cb4e58aeeb191fdde9e99c/users/1b264574101c474f976f6e0c7857b5ff/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:47.081240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0af98-354c-45d9-acda-d583fb3b9fbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc42dc9a609d441e848320a89013ad26, user_id=e0b603469c0f4d4fa5a332aec545aca9, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:47.084823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b0af98-354c-45d9-acda-d583fb3b9fbd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:47.089899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207b5e3-5211-4193-85ab-335783e2cd49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6268f144e4cb4e58aeeb191fdde9e99c, user_id=1b264574101c474f976f6e0c7857b5ff, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:47.095336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5207b5e3-5211-4193-85ab-335783e2cd49 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:47.118437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 977/1937] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:47 2022] PUT /identity/v3/projects/6268f144e4cb4e58aeeb191fdde9e99c/users/1b264574101c474f976f6e0c7857b5ff/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:47.119196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207b5e3-5211-4193-85ab-335783e2cd49 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:47.121037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5207b5e3-5211-4193-85ab-335783e2cd49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:47.124224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 961/1938] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:46 2022] PUT /identity/v3/projects/cc42dc9a609d441e848320a89013ad26/users/e0b603469c0f4d4fa5a332aec545aca9/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:47.130805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e17cbc-03d8-409c-bca7-de94b9f67bd1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:47.131448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e17cbc-03d8-409c-bca7-de94b9f67bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:47.131940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e17cbc-03d8-409c-bca7-de94b9f67bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:47.201440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f6e17cbc-03d8-409c-bca7-de94b9f67bd1 None None] MFA Rules not processed for user `1b264574101c474f976f6e0c7857b5ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:47.203161 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f6e17cbc-03d8-409c-bca7-de94b9f67bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:47.268355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5207b5e3-5211-4193-85ab-335783e2cd49 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:47.279548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:47.279548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:47.279548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:47.279548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] PATH_INFO: `/v3/projects/35ab8e4d17ba43c0ab828fcbd7593931` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:47.280034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] RBAC: Authorizing `identity:delete_project(project_id=35ab8e4d17ba43c0ab828fcbd7593931)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:47.282789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:47.311145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 962/1939] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:47.313648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab42b6e-16aa-4689-9d8d-33df4975c7af None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:47.314264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab42b6e-16aa-4689-9d8d-33df4975c7af None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:47.314583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab42b6e-16aa-4689-9d8d-33df4975c7af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:47.357982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1ab42b6e-16aa-4689-9d8d-33df4975c7af None None] MFA Rules not processed for user `e0b603469c0f4d4fa5a332aec545aca9`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:47.360005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1ab42b6e-16aa-4689-9d8d-33df4975c7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:47.413429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35ab8e4d17ba43c0ab828fcbd7593931', 'request_id': 'req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:47.433237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35ab8e4d17ba43c0ab828fcbd7593931', 'request_id': 'req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:47.442456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:47.452213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35ab8e4d17ba43c0ab828fcbd7593931', 'request_id': 'req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:47.475147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] The token cache is being invalidate because project 35ab8e4d17ba43c0ab828fcbd7593931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:47.475147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:47.475147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:47.477272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 978/1940] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:47 2022] DELETE /identity/v3/projects/35ab8e4d17ba43c0ab828fcbd7593931 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:47.478262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:47.480492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:47.532680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:47.535432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 963/1941] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:47.536957 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab42b6e-16aa-4689-9d8d-33df4975c7af None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:47.538445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1ab42b6e-16aa-4689-9d8d-33df4975c7af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:47.641736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e404b1e9-abd2-4fbd-a6c4-969e3391c33a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:47.651251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab42b6e-16aa-4689-9d8d-33df4975c7af None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:47.655299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7777b-e182-4c86-8861-2b1ae6074849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:47.657580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7777b-e182-4c86-8861-2b1ae6074849 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:47.659013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7777b-e182-4c86-8861-2b1ae6074849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:47.659197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:47.659561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d7777b-e182-4c86-8861-2b1ae6074849 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:47.660017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:47.662281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7777b-e182-4c86-8861-2b1ae6074849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:47.667675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:47.668214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] PATH_INFO: `/v3/users/adcf9cf7f6c742be876d183e7e17c7d5` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:47.670304 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adcf9cf7f6c742be876d183e7e17c7d5)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:47.670911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d7777b-e182-4c86-8861-2b1ae6074849 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:47.673136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:47.725366 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 979/1942] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:47 2022] POST /identity/v3/projects => generated 389 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:47.726779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7777b-e182-4c86-8861-2b1ae6074849 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:47.729394 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-84d7777b-e182-4c86-8861-2b1ae6074849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:47.772382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adcf9cf7f6c742be876d183e7e17c7d5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:47.773101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adcf9cf7f6c742be876d183e7e17c7d5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:47.785592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adcf9cf7f6c742be876d183e7e17c7d5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:47.799772 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adcf9cf7f6c742be876d183e7e17c7d5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:47.816980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adcf9cf7f6c742be876d183e7e17c7d5'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:47.818291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d7777b-e182-4c86-8861-2b1ae6074849 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:47.822975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bcd28-cfa1-4a77-8310-6cdfece7d628 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:47.823723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bcd28-cfa1-4a77-8310-6cdfece7d628 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:47.824105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bcd28-cfa1-4a77-8310-6cdfece7d628 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:47.824227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 964/1943] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:47 2022] DELETE /identity/v3/users/adcf9cf7f6c742be876d183e7e17c7d5 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:47.824666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81bcd28-cfa1-4a77-8310-6cdfece7d628 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:47.825441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:47.826791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:47.826909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f81bcd28-cfa1-4a77-8310-6cdfece7d628 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:47.900333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:47.925557 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81bcd28-cfa1-4a77-8310-6cdfece7d628 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:47.927050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81bcd28-cfa1-4a77-8310-6cdfece7d628 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:47.933131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af58811-5bbd-48bb-85f0-d626ef6d33c3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:47.937307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 980/1944] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:47.938829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3df2-5835-4628-8eb6-0464449eb440 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:47.939613 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3df2-5835-4628-8eb6-0464449eb440 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:47.940013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3df2-5835-4628-8eb6-0464449eb440 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:47.940693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1a3df2-5835-4628-8eb6-0464449eb440 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:47.941643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a3df2-5835-4628-8eb6-0464449eb440 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:47.943802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1a3df2-5835-4628-8eb6-0464449eb440 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:47.974773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 965/1945] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:47 2022] POST /identity/v3/users => generated 301 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:47.975866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bcd28-cfa1-4a77-8310-6cdfece7d628 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:48.091478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81bcd28-cfa1-4a77-8310-6cdfece7d628 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:48.097927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331f73-f93f-4bd5-a0f9-064588bcaae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:48.098807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331f73-f93f-4bd5-a0f9-064588bcaae0 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:48.099330 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331f73-f93f-4bd5-a0f9-064588bcaae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:48.099748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1331f73-f93f-4bd5-a0f9-064588bcaae0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:48.100694 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1331f73-f93f-4bd5-a0f9-064588bcaae0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:48.102709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1331f73-f93f-4bd5-a0f9-064588bcaae0 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:48.116860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 981/1946] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:47 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:48.122105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3df2-5835-4628-8eb6-0464449eb440 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:48.134791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1a3df2-5835-4628-8eb6-0464449eb440 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:48.167207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1829f-8bfc-415b-a1b1-5d0a3bf9abcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:48.167207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1829f-8bfc-415b-a1b1-5d0a3bf9abcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:48.167207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1829f-8bfc-415b-a1b1-5d0a3bf9abcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:48.167590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1829f-8bfc-415b-a1b1-5d0a3bf9abcd admin admin] PATH_INFO: `/v3/projects/e81a478bd55d40fcacfa595091e0ae4b/users/1e5795b3a80e4503afb2aae71cac7d4a/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:48.196112 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1829f-8bfc-415b-a1b1-5d0a3bf9abcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81a478bd55d40fcacfa595091e0ae4b, user_id=1e5795b3a80e4503afb2aae71cac7d4a, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:48.200365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1829f-8bfc-415b-a1b1-5d0a3bf9abcd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:48.230688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 966/1947] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:48 2022] PUT /identity/v3/projects/e81a478bd55d40fcacfa595091e0ae4b/users/1e5795b3a80e4503afb2aae71cac7d4a/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:48.233132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331f73-f93f-4bd5-a0f9-064588bcaae0 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:48.259933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1331f73-f93f-4bd5-a0f9-064588bcaae0 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:48.266078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:48.335866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eddd983-21e2-4748-ab7c-4665a83766ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:48.336678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddd983-21e2-4748-ab7c-4665a83766ca admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:48.337081 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddd983-21e2-4748-ab7c-4665a83766ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:48.337532 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eddd983-21e2-4748-ab7c-4665a83766ca admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:48.338375 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddd983-21e2-4748-ab7c-4665a83766ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:48.340293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eddd983-21e2-4748-ab7c-4665a83766ca admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:48.354023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 982/1948] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:48 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:48.358943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1829f-8bfc-415b-a1b1-5d0a3bf9abcd admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:48.377719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1829f-8bfc-415b-a1b1-5d0a3bf9abcd admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:48.389692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5abe9c-c5bb-44f7-8a2b-b02c6dffe74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:48.390379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5abe9c-c5bb-44f7-8a2b-b02c6dffe74f admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:48.391084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5abe9c-c5bb-44f7-8a2b-b02c6dffe74f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:48.392639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5abe9c-c5bb-44f7-8a2b-b02c6dffe74f admin admin] PATH_INFO: `/v3/projects/e81a478bd55d40fcacfa595091e0ae4b/users/1e5795b3a80e4503afb2aae71cac7d4a/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:48.398358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5abe9c-c5bb-44f7-8a2b-b02c6dffe74f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e81a478bd55d40fcacfa595091e0ae4b, user_id=1e5795b3a80e4503afb2aae71cac7d4a, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:48.403171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5abe9c-c5bb-44f7-8a2b-b02c6dffe74f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:48.447957 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 967/1949] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:48 2022] PUT /identity/v3/projects/e81a478bd55d40fcacfa595091e0ae4b/users/1e5795b3a80e4503afb2aae71cac7d4a/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:48.459031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c347ca1-6a58-4555-a4de-b6d29056ea93 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:48.459584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c347ca1-6a58-4555-a4de-b6d29056ea93 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:48.460010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c347ca1-6a58-4555-a4de-b6d29056ea93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:48.495772 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-9c347ca1-6a58-4555-a4de-b6d29056ea93 None None] MFA Rules not processed for user `1e5795b3a80e4503afb2aae71cac7d4a`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:48.497009 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9c347ca1-6a58-4555-a4de-b6d29056ea93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:48.589008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 983/1950] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:48 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:48.605765 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5abe9c-c5bb-44f7-8a2b-b02c6dffe74f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:48.608559 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3c5abe9c-c5bb-44f7-8a2b-b02c6dffe74f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:48.633671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c347ca1-6a58-4555-a4de-b6d29056ea93 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:48.635441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9c347ca1-6a58-4555-a4de-b6d29056ea93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:48.702102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5abe9c-c5bb-44f7-8a2b-b02c6dffe74f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:48.709794 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbea7a61-427c-4fdf-8259-26e615936769 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:48.710431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea7a61-427c-4fdf-8259-26e615936769 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:48.720178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea7a61-427c-4fdf-8259-26e615936769 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:48.720463 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbea7a61-427c-4fdf-8259-26e615936769 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:48.732501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c347ca1-6a58-4555-a4de-b6d29056ea93 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:48.732880 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea7a61-427c-4fdf-8259-26e615936769 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:48.736283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbea7a61-427c-4fdf-8259-26e615936769 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:48.741745 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:48.742522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:48.743813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:48.744220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] PATH_INFO: `/v3/projects/a37f171018c04854bd99a88526d5b44f` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:48.746279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a37f171018c04854bd99a88526d5b44f)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:48.748983 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:48.749931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 968/1951] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:48 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:48.805765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:48.826350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a37f171018c04854bd99a88526d5b44f', 'request_id': 'req-40c31176-a357-4c79-a3cc-048bd8134d6b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:48.841354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a37f171018c04854bd99a88526d5b44f', 'request_id': 'req-40c31176-a357-4c79-a3cc-048bd8134d6b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:48.853606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a37f171018c04854bd99a88526d5b44f', 'request_id': 'req-40c31176-a357-4c79-a3cc-048bd8134d6b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:48.882712 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] The token cache is being invalidate because project a37f171018c04854bd99a88526d5b44f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:48.883176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:48.883926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:48.894266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd4f1b-a929-471f-b11f-fe2c5acf9e0d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:48.894557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd4f1b-a929-471f-b11f-fe2c5acf9e0d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:48.894846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cd4f1b-a929-471f-b11f-fe2c5acf9e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:48.899046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 984/1952] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:48 2022] DELETE /identity/v3/projects/a37f171018c04854bd99a88526d5b44f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:48.944408 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-12cd4f1b-a929-471f-b11f-fe2c5acf9e0d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:48.949490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-12cd4f1b-a929-471f-b11f-fe2c5acf9e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:49.018623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:49.020524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:49.113611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 969/1953] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:49.122847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0f8e7-1d0e-4539-a463-2d07197e2a5c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:49.122847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0f8e7-1d0e-4539-a463-2d07197e2a5c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:49.123124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0f8e7-1d0e-4539-a463-2d07197e2a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:49.158064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-faa0f8e7-1d0e-4539-a463-2d07197e2a5c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:49.160167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-faa0f8e7-1d0e-4539-a463-2d07197e2a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:49.166525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c31176-a357-4c79-a3cc-048bd8134d6b admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:49.170013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 970/1954] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:49.177211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0f8e7-1d0e-4539-a463-2d07197e2a5c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:49.179925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb5086b-6fa4-4eaa-81fe-f80a3594d0ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:49.180581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5086b-6fa4-4eaa-81fe-f80a3594d0ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:49.180907 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5086b-6fa4-4eaa-81fe-f80a3594d0ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:49.181180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb5086b-6fa4-4eaa-81fe-f80a3594d0ee admin admin] PATH_INFO: `/v3/users/1b264574101c474f976f6e0c7857b5ff` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:49.182389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5086b-6fa4-4eaa-81fe-f80a3594d0ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b264574101c474f976f6e0c7857b5ff)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:49.184316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb5086b-6fa4-4eaa-81fe-f80a3594d0ee admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:49.190675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0f8e7-1d0e-4539-a463-2d07197e2a5c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:49.195522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc75ec-dfe9-4c8f-913b-3cc6a3083a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:49.196269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc75ec-dfe9-4c8f-913b-3cc6a3083a0c admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:49.196532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc75ec-dfe9-4c8f-913b-3cc6a3083a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:49.197556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0bc75ec-dfe9-4c8f-913b-3cc6a3083a0c admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:49.198344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc75ec-dfe9-4c8f-913b-3cc6a3083a0c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:49.198647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc75ec-dfe9-4c8f-913b-3cc6a3083a0c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:49.200295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0bc75ec-dfe9-4c8f-913b-3cc6a3083a0c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:49.215913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 971/1955] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:43:49.220504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc75ec-dfe9-4c8f-913b-3cc6a3083a0c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:49.232037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0bc75ec-dfe9-4c8f-913b-3cc6a3083a0c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:49.237719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ce229-cd46-44ca-8871-60bfe2a2235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:49.238686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ce229-cd46-44ca-8871-60bfe2a2235a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:49.238872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ce229-cd46-44ca-8871-60bfe2a2235a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:49.239240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7ce229-cd46-44ca-8871-60bfe2a2235a admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:49.239946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ce229-cd46-44ca-8871-60bfe2a2235a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:49.241994 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7ce229-cd46-44ca-8871-60bfe2a2235a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:49.257695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-feb5086b-6fa4-4eaa-81fe-f80a3594d0ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b264574101c474f976f6e0c7857b5ff'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:49.271593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:49.272019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-feb5086b-6fa4-4eaa-81fe-f80a3594d0ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b264574101c474f976f6e0c7857b5ff'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:49.272874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-feb5086b-6fa4-4eaa-81fe-f80a3594d0ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b264574101c474f976f6e0c7857b5ff'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:49.301526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 972/1956] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:49 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:49.302435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ce229-cd46-44ca-8871-60bfe2a2235a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:49.303398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-feb5086b-6fa4-4eaa-81fe-f80a3594d0ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b264574101c474f976f6e0c7857b5ff'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:49.305411 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1d7ce229-cd46-44ca-8871-60bfe2a2235a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:49.313795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-feb5086b-6fa4-4eaa-81fe-f80a3594d0ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b264574101c474f976f6e0c7857b5ff'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:49.322252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 985/1957] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:49 2022] DELETE /identity/v3/users/1b264574101c474f976f6e0c7857b5ff => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:49.326805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6378c3-5245-4334-b939-a0fa61ab4d93 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:49.327486 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6378c3-5245-4334-b939-a0fa61ab4d93 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:49.328010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6378c3-5245-4334-b939-a0fa61ab4d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:49.357918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-dd6378c3-5245-4334-b939-a0fa61ab4d93 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:49.360191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-dd6378c3-5245-4334-b939-a0fa61ab4d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:49.410932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7ce229-cd46-44ca-8871-60bfe2a2235a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:49.411939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:49.459352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 986/1958] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:49.475128 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6378c3-5245-4334-b939-a0fa61ab4d93 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:49.476546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-dd6378c3-5245-4334-b939-a0fa61ab4d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:49.513037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20d0ad-34a2-4793-90d8-cf1501bf94cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:49.513760 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20d0ad-34a2-4793-90d8-cf1501bf94cd admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:49.514152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20d0ad-34a2-4793-90d8-cf1501bf94cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:49.514521 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc20d0ad-34a2-4793-90d8-cf1501bf94cd admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:49.515546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20d0ad-34a2-4793-90d8-cf1501bf94cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:49.518190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc20d0ad-34a2-4793-90d8-cf1501bf94cd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:49.585559 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 973/1959] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:49 2022] POST /identity/v3/users => generated 293 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:49.587129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20d0ad-34a2-4793-90d8-cf1501bf94cd admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:49.613667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc20d0ad-34a2-4793-90d8-cf1501bf94cd admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:49.619900 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6378c3-5245-4334-b939-a0fa61ab4d93 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:49.625868 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76efd7-4c7f-43bb-853f-8e5beb17be67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:49.626467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76efd7-4c7f-43bb-853f-8e5beb17be67 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:49.627928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d20e46-da70-421c-af82-de0595c4b3e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:49.628885 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76efd7-4c7f-43bb-853f-8e5beb17be67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:49.629064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20e46-da70-421c-af82-de0595c4b3e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:49.629244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c76efd7-4c7f-43bb-853f-8e5beb17be67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:49.629420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20e46-da70-421c-af82-de0595c4b3e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:49.629699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d20e46-da70-421c-af82-de0595c4b3e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:49.629874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76efd7-4c7f-43bb-853f-8e5beb17be67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:49.631836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c76efd7-4c7f-43bb-853f-8e5beb17be67 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:49.642594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d20e46-da70-421c-af82-de0595c4b3e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:49.644656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d20e46-da70-421c-af82-de0595c4b3e6 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:49.649585 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 974/1960] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:49.653896 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76efd7-4c7f-43bb-853f-8e5beb17be67 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:49.660873 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 987/1961] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:43:49.670086 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c76efd7-4c7f-43bb-853f-8e5beb17be67 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:49.676175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90792ac-1a6e-4d2b-99d0-83b081643ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:49.676797 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90792ac-1a6e-4d2b-99d0-83b081643ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:49.677091 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90792ac-1a6e-4d2b-99d0-83b081643ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:49.677543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90792ac-1a6e-4d2b-99d0-83b081643ce5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:49.678328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90792ac-1a6e-4d2b-99d0-83b081643ce5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:49.680279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90792ac-1a6e-4d2b-99d0-83b081643ce5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:49.694969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 975/1962] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:49 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:49.698871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d20e46-da70-421c-af82-de0595c4b3e6 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:49.713601 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d20e46-da70-421c-af82-de0595c4b3e6 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:49.722230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69501d40-2fb7-47d5-98e1-129c8b531ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:49.723129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69501d40-2fb7-47d5-98e1-129c8b531ab6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:49.723467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69501d40-2fb7-47d5-98e1-129c8b531ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:49.723829 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69501d40-2fb7-47d5-98e1-129c8b531ab6 admin admin] PATH_INFO: `/v3/projects/aac81aafc28f4bd29a20fe59ed85ca3a/users/9653434268634d11816643a9ac1a86f0/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:49.747833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69501d40-2fb7-47d5-98e1-129c8b531ab6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aac81aafc28f4bd29a20fe59ed85ca3a, user_id=9653434268634d11816643a9ac1a86f0, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:49.751625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69501d40-2fb7-47d5-98e1-129c8b531ab6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:49.776409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 988/1963] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:49 2022] PUT /identity/v3/projects/aac81aafc28f4bd29a20fe59ed85ca3a/users/9653434268634d11816643a9ac1a86f0/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:49.787758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b03b9-c618-492e-9f89-c38bb63b9b72 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:49.788140 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b03b9-c618-492e-9f89-c38bb63b9b72 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:49.788508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864b03b9-c618-492e-9f89-c38bb63b9b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:49.831313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-864b03b9-c618-492e-9f89-c38bb63b9b72 None None] MFA Rules not processed for user `9653434268634d11816643a9ac1a86f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:49.831313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-864b03b9-c618-492e-9f89-c38bb63b9b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:49.912314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:49.967150 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 976/1964] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:49.967525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69501d40-2fb7-47d5-98e1-129c8b531ab6 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:50.059313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b03b9-c618-492e-9f89-c38bb63b9b72 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:50.061496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-864b03b9-c618-492e-9f89-c38bb63b9b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:50.092526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69501d40-2fb7-47d5-98e1-129c8b531ab6 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:50.098216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b9d0a-0e9b-4ff0-ac2b-370ef910cba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:50.099104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b9d0a-0e9b-4ff0-ac2b-370ef910cba8 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:50.099823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b9d0a-0e9b-4ff0-ac2b-370ef910cba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:50.100248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58b9d0a-0e9b-4ff0-ac2b-370ef910cba8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:50.101078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b9d0a-0e9b-4ff0-ac2b-370ef910cba8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:50.104586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58b9d0a-0e9b-4ff0-ac2b-370ef910cba8 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:50.153491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 989/1965] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:49 2022] POST /identity/v3/projects => generated 385 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:50.157342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b9d0a-0e9b-4ff0-ac2b-370ef910cba8 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:50.236690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864b03b9-c618-492e-9f89-c38bb63b9b72 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:50.239605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:50.267507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58b9d0a-0e9b-4ff0-ac2b-370ef910cba8 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:50.274415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa8530-225a-4584-bdfd-22acfed7e87f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:50.275359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa8530-225a-4584-bdfd-22acfed7e87f admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:50.276468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa8530-225a-4584-bdfd-22acfed7e87f admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:50.276854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa8530-225a-4584-bdfd-22acfed7e87f admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:50.278021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa8530-225a-4584-bdfd-22acfed7e87f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:50.282049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa8530-225a-4584-bdfd-22acfed7e87f admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:50.318009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:50.318658 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:50.318942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:50.319212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] PATH_INFO: `/v3/projects/6268f144e4cb4e58aeeb191fdde9e99c` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:50.320350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6268f144e4cb4e58aeeb191fdde9e99c)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:50.322288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:50.329098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 990/1966] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:50 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:50.329342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa8530-225a-4584-bdfd-22acfed7e87f admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:50.341356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa8530-225a-4584-bdfd-22acfed7e87f admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:50.346518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600ef6-db58-45ba-93cb-67be8b796e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:50.347143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600ef6-db58-45ba-93cb-67be8b796e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:50.347576 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600ef6-db58-45ba-93cb-67be8b796e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:50.347852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18600ef6-db58-45ba-93cb-67be8b796e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:50.348499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18600ef6-db58-45ba-93cb-67be8b796e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:50.350115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18600ef6-db58-45ba-93cb-67be8b796e63 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:50.363322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 991/1967] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:43:50.367143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600ef6-db58-45ba-93cb-67be8b796e63 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:50.384063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18600ef6-db58-45ba-93cb-67be8b796e63 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:50.394820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6268f144e4cb4e58aeeb191fdde9e99c', 'request_id': 'req-2e0fe235-a7e8-46cc-98e1-36ea266d235b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:50.408374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6268f144e4cb4e58aeeb191fdde9e99c', 'request_id': 'req-2e0fe235-a7e8-46cc-98e1-36ea266d235b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:50.420702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6268f144e4cb4e58aeeb191fdde9e99c', 'request_id': 'req-2e0fe235-a7e8-46cc-98e1-36ea266d235b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:50.442119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] The token cache is being invalidate because project 6268f144e4cb4e58aeeb191fdde9e99c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:50.442510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:50.443320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:50.445305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 977/1968] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:50 2022] DELETE /identity/v3/projects/6268f144e4cb4e58aeeb191fdde9e99c => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:50.449695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:50.451288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:50.467731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e0bc4-3a72-41f0-9714-65be2fe7d64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:50.468521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e0bc4-3a72-41f0-9714-65be2fe7d64f admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:50.469546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e0bc4-3a72-41f0-9714-65be2fe7d64f admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:50.469996 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532e0bc4-3a72-41f0-9714-65be2fe7d64f admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:50.470744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e0bc4-3a72-41f0-9714-65be2fe7d64f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:50.472517 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532e0bc4-3a72-41f0-9714-65be2fe7d64f admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:50.486032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 992/1969] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:50 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:50.490078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e0bc4-3a72-41f0-9714-65be2fe7d64f admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:50.491800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-532e0bc4-3a72-41f0-9714-65be2fe7d64f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:50.498815 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:50.508702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532e0bc4-3a72-41f0-9714-65be2fe7d64f admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:50.513879 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b10f58-8b5e-4b8a-ad8e-aa84660f3060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:50.514552 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b10f58-8b5e-4b8a-ad8e-aa84660f3060 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:50.514828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b10f58-8b5e-4b8a-ad8e-aa84660f3060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:50.515125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b10f58-8b5e-4b8a-ad8e-aa84660f3060 admin admin] PATH_INFO: `/v3/projects/e34df43a48614f608e343e1250612372/users/d8bda835e87c40549746538fe48812ef/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:50.531656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b10f58-8b5e-4b8a-ad8e-aa84660f3060 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e34df43a48614f608e343e1250612372, user_id=d8bda835e87c40549746538fe48812ef, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:50.534637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b10f58-8b5e-4b8a-ad8e-aa84660f3060 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:50.540191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0fe235-a7e8-46cc-98e1-36ea266d235b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:50.547898 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:50.548693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:50.549003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:50.549269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] PATH_INFO: `/v3/users/1e5795b3a80e4503afb2aae71cac7d4a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:50.549979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 993/1970] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:50 2022] PUT /identity/v3/projects/e34df43a48614f608e343e1250612372/users/d8bda835e87c40549746538fe48812ef/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:50.550541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e5795b3a80e4503afb2aae71cac7d4a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:50.552453 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:50.561258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556ea4-e793-4a2d-8cae-8da29851b642 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:50.561823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556ea4-e793-4a2d-8cae-8da29851b642 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:50.562303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2556ea4-e793-4a2d-8cae-8da29851b642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:50.602129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c2556ea4-e793-4a2d-8cae-8da29851b642 None None] MFA Rules not processed for user `d8bda835e87c40549746538fe48812ef`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:50.603435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c2556ea4-e793-4a2d-8cae-8da29851b642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:50.613483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e5795b3a80e4503afb2aae71cac7d4a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:50.614387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e5795b3a80e4503afb2aae71cac7d4a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:50.622044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e5795b3a80e4503afb2aae71cac7d4a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:50.630718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e5795b3a80e4503afb2aae71cac7d4a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:50.645681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e5795b3a80e4503afb2aae71cac7d4a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:50.655135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 978/1971] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:50 2022] DELETE /identity/v3/users/1e5795b3a80e4503afb2aae71cac7d4a => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:50.687373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 994/1972] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:50 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:50.689263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:50.691581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:50.776511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63dedb0e-f765-4dd1-ac01-8986a837abaf admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:50.781138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752c211-4c83-4a24-be4e-44e23acefd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:50.781924 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752c211-4c83-4a24-be4e-44e23acefd90 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:50.782307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752c211-4c83-4a24-be4e-44e23acefd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:50.782684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4752c211-4c83-4a24-be4e-44e23acefd90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:50.783368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752c211-4c83-4a24-be4e-44e23acefd90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:50.785552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4752c211-4c83-4a24-be4e-44e23acefd90 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:50.820092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 979/1973] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:50 2022] POST /identity/v3/projects => generated 385 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:50.824870 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2556ea4-e793-4a2d-8cae-8da29851b642 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:50.870402 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:50.915333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2556ea4-e793-4a2d-8cae-8da29851b642 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:50.950443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752c211-4c83-4a24-be4e-44e23acefd90 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:50.952350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4752c211-4c83-4a24-be4e-44e23acefd90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:51.008117 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be7e69-bd51-405d-b2e0-f90bc6416120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.008731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7e69-bd51-405d-b2e0-f90bc6416120 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.009035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7e69-bd51-405d-b2e0-f90bc6416120 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.009329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7e69-bd51-405d-b2e0-f90bc6416120 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.010094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be7e69-bd51-405d-b2e0-f90bc6416120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.010635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:51.012076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be7e69-bd51-405d-b2e0-f90bc6416120 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.037614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 995/1974] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:50 2022] POST /identity/v3/users => generated 299 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:51.039526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be7e69-bd51-405d-b2e0-f90bc6416120 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.042046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4752c211-4c83-4a24-be4e-44e23acefd90 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.047285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.048121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.048391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.048655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] PATH_INFO: `/v3/projects/e81a478bd55d40fcacfa595091e0ae4b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.049064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be7e69-bd51-405d-b2e0-f90bc6416120 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.053179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e81a478bd55d40fcacfa595091e0ae4b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.055502 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.057011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b45854-4426-437d-ac2f-46f259ae1b5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.057622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b45854-4426-437d-ac2f-46f259ae1b5a admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.057895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b45854-4426-437d-ac2f-46f259ae1b5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.058163 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b45854-4426-437d-ac2f-46f259ae1b5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.058794 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b45854-4426-437d-ac2f-46f259ae1b5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.060388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b45854-4426-437d-ac2f-46f259ae1b5a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.071012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 996/1975] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:51.075152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b45854-4426-437d-ac2f-46f259ae1b5a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.085354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b45854-4426-437d-ac2f-46f259ae1b5a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.090083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9058761-393e-43fa-90bd-3af529c6b080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.090718 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9058761-393e-43fa-90bd-3af529c6b080 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.090991 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9058761-393e-43fa-90bd-3af529c6b080 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.091254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9058761-393e-43fa-90bd-3af529c6b080 admin admin] PATH_INFO: `/v3/projects/83ea20527bf34830b7ac0646279d526a/users/1824f98e9d6e4ed4938d1e14270c0b53/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.104248 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e81a478bd55d40fcacfa595091e0ae4b', 'request_id': 'req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:51.107244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9058761-393e-43fa-90bd-3af529c6b080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ea20527bf34830b7ac0646279d526a, user_id=1824f98e9d6e4ed4938d1e14270c0b53, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.110983 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9058761-393e-43fa-90bd-3af529c6b080 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.111363 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e81a478bd55d40fcacfa595091e0ae4b', 'request_id': 'req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:51.119222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e81a478bd55d40fcacfa595091e0ae4b', 'request_id': 'req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:51.124794 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 997/1976] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:51 2022] PUT /identity/v3/projects/83ea20527bf34830b7ac0646279d526a/users/1824f98e9d6e4ed4938d1e14270c0b53/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:51.129330 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9058761-393e-43fa-90bd-3af529c6b080 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.131006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] The token cache is being invalidate because project e81a478bd55d40fcacfa595091e0ae4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:51.131303 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:51.131952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-ea71b539-1297-4a0e-8a1c-cd4ca4bc00d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:51.133500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 980/1977] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:50 2022] DELETE /identity/v3/projects/e81a478bd55d40fcacfa595091e0ae4b => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:51.138389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a9f41-6664-412c-8efc-bb90517e5cb3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.138676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a9f41-6664-412c-8efc-bb90517e5cb3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.138948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a9f41-6664-412c-8efc-bb90517e5cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.159614 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c87a9f41-6664-412c-8efc-bb90517e5cb3 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:51.160843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c87a9f41-6664-412c-8efc-bb90517e5cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:51.191769 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9058761-393e-43fa-90bd-3af529c6b080 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.196354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9f5c3-7aed-463f-960a-d25c922953b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.196988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9f5c3-7aed-463f-960a-d25c922953b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.197265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9f5c3-7aed-463f-960a-d25c922953b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.197537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff9f5c3-7aed-463f-960a-d25c922953b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.198168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9f5c3-7aed-463f-960a-d25c922953b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.199746 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff9f5c3-7aed-463f-960a-d25c922953b0 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.210324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 998/1978] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:51.214506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9f5c3-7aed-463f-960a-d25c922953b0 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.216140 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-fff9f5c3-7aed-463f-960a-d25c922953b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:51.220992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 981/1979] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:51.228346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f714da6-899a-46c4-9b2d-92aadf8367d3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.228624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f714da6-899a-46c4-9b2d-92aadf8367d3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.228929 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f714da6-899a-46c4-9b2d-92aadf8367d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.229374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff9f5c3-7aed-463f-960a-d25c922953b0 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.233984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031868ad-5a32-46cb-b92e-ce6146ee8d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.234668 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031868ad-5a32-46cb-b92e-ce6146ee8d94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.234950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031868ad-5a32-46cb-b92e-ce6146ee8d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.235214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031868ad-5a32-46cb-b92e-ce6146ee8d94 admin admin] PATH_INFO: `/v3/projects/83ea20527bf34830b7ac0646279d526a/users/1824f98e9d6e4ed4938d1e14270c0b53/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.237282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031868ad-5a32-46cb-b92e-ce6146ee8d94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=83ea20527bf34830b7ac0646279d526a, user_id=1824f98e9d6e4ed4938d1e14270c0b53, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.239920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031868ad-5a32-46cb-b92e-ce6146ee8d94 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.249314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6f714da6-899a-46c4-9b2d-92aadf8367d3 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:51.250540 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6f714da6-899a-46c4-9b2d-92aadf8367d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:51.250868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 999/1980] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:51 2022] PUT /identity/v3/projects/83ea20527bf34830b7ac0646279d526a/users/1824f98e9d6e4ed4938d1e14270c0b53/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:51.261411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c961-fa82-40a9-aeb0-d8697f484e5b None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.261692 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c961-fa82-40a9-aeb0-d8697f484e5b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.261962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39c961-fa82-40a9-aeb0-d8697f484e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.294475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:51.299875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-fc39c961-fa82-40a9-aeb0-d8697f484e5b None None] MFA Rules not processed for user `1824f98e9d6e4ed4938d1e14270c0b53`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:51.302558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-fc39c961-fa82-40a9-aeb0-d8697f484e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:51.330138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 982/1981] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:51.334760 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f714da6-899a-46c4-9b2d-92aadf8367d3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.343899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f714da6-899a-46c4-9b2d-92aadf8367d3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.348871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609999b1-7553-4951-9898-a323cfbfec98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.349496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609999b1-7553-4951-9898-a323cfbfec98 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.349774 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609999b1-7553-4951-9898-a323cfbfec98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.350041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609999b1-7553-4951-9898-a323cfbfec98 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.350684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609999b1-7553-4951-9898-a323cfbfec98 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:51.350981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609999b1-7553-4951-9898-a323cfbfec98 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.352605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609999b1-7553-4951-9898-a323cfbfec98 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.355217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:51.380285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 983/1982] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:43:51.385181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1000/1983] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:51 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:51.385334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609999b1-7553-4951-9898-a323cfbfec98 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.393650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39c961-fa82-40a9-aeb0-d8697f484e5b None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.393884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609999b1-7553-4951-9898-a323cfbfec98 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.395280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-fc39c961-fa82-40a9-aeb0-d8697f484e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:51.398761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3225f0-d94b-4087-b5bd-ad58cb2beae9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.399358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3225f0-d94b-4087-b5bd-ad58cb2beae9 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.399636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3225f0-d94b-4087-b5bd-ad58cb2beae9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.399904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3225f0-d94b-4087-b5bd-ad58cb2beae9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.400607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3225f0-d94b-4087-b5bd-ad58cb2beae9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.402602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3225f0-d94b-4087-b5bd-ad58cb2beae9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.435134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 984/1984] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:51 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:51.436746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3225f0-d94b-4087-b5bd-ad58cb2beae9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.453494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39c961-fa82-40a9-aeb0-d8697f484e5b None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.507558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d79b2-8d89-47bc-b45c-c8f4615283ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.508190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d79b2-8d89-47bc-b45c-c8f4615283ec service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.508462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d79b2-8d89-47bc-b45c-c8f4615283ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.508674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3225f0-d94b-4087-b5bd-ad58cb2beae9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.508820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64d79b2-8d89-47bc-b45c-c8f4615283ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.510575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d64d79b2-8d89-47bc-b45c-c8f4615283ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:51.513757 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae11b4-fae5-4331-a41b-28829061f45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.514373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae11b4-fae5-4331-a41b-28829061f45c admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.514651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae11b4-fae5-4331-a41b-28829061f45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.514917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ae11b4-fae5-4331-a41b-28829061f45c admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.515586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae11b4-fae5-4331-a41b-28829061f45c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.517818 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ae11b4-fae5-4331-a41b-28829061f45c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.552802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 985/1985] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:51 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:51.554354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae11b4-fae5-4331-a41b-28829061f45c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.567266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ae11b4-fae5-4331-a41b-28829061f45c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.578815 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d95a6d-a04d-420c-b56b-9917630ffe1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.580041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d95a6d-a04d-420c-b56b-9917630ffe1e admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.580041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d95a6d-a04d-420c-b56b-9917630ffe1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.580197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d95a6d-a04d-420c-b56b-9917630ffe1e admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.580730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d95a6d-a04d-420c-b56b-9917630ffe1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.582386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d95a6d-a04d-420c-b56b-9917630ffe1e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.585456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d79b2-8d89-47bc-b45c-c8f4615283ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.587053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64d79b2-8d89-47bc-b45c-c8f4615283ec service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.595063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 986/1986] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:51.597884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1001/1987] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:51 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:43:51.598939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d95a6d-a04d-420c-b56b-9917630ffe1e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.608850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d95a6d-a04d-420c-b56b-9917630ffe1e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.615846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56671e70-bc2e-49a7-bc49-294f724bfed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.616520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56671e70-bc2e-49a7-bc49-294f724bfed4 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.616869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56671e70-bc2e-49a7-bc49-294f724bfed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.617149 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56671e70-bc2e-49a7-bc49-294f724bfed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.618903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56671e70-bc2e-49a7-bc49-294f724bfed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.619445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56671e70-bc2e-49a7-bc49-294f724bfed4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.640317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 987/1988] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:43:51.645525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d79b2-8d89-47bc-b45c-c8f4615283ec service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.653292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:51.658515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64d79b2-8d89-47bc-b45c-c8f4615283ec service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.663695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066b841-ad58-4b8c-9748-7d9a669dc4ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.664349 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066b841-ad58-4b8c-9748-7d9a669dc4ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.664625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066b841-ad58-4b8c-9748-7d9a669dc4ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.664933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a066b841-ad58-4b8c-9748-7d9a669dc4ee admin admin] PATH_INFO: `/v3/projects/19855dafaf8542f0a58d8d0467503e88/users/3eebc564cb2a4b26bdcf6cf3fd2cd744/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.681280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066b841-ad58-4b8c-9748-7d9a669dc4ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=19855dafaf8542f0a58d8d0467503e88, user_id=3eebc564cb2a4b26bdcf6cf3fd2cd744, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.683914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a066b841-ad58-4b8c-9748-7d9a669dc4ee admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.698550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1002/1989] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:51 2022] PUT /identity/v3/projects/19855dafaf8542f0a58d8d0467503e88/users/3eebc564cb2a4b26bdcf6cf3fd2cd744/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:51.704454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a0db-288e-4955-98be-437a52e93201 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.704622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a0db-288e-4955-98be-437a52e93201 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.704870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a0a0db-288e-4955-98be-437a52e93201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.723437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d8a0a0db-288e-4955-98be-437a52e93201 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:51.724626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d8a0a0db-288e-4955-98be-437a52e93201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:51.790675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 988/1990] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:51.800931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066b841-ad58-4b8c-9748-7d9a669dc4ee admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:51.802874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a066b841-ad58-4b8c-9748-7d9a669dc4ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:51.867669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a066b841-ad58-4b8c-9748-7d9a669dc4ee admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:51.871473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd33dc-e800-4b56-9ab8-dfff8013049a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:51.872090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd33dc-e800-4b56-9ab8-dfff8013049a service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:51.872548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd33dc-e800-4b56-9ab8-dfff8013049a service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:51.872900 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd33dc-e800-4b56-9ab8-dfff8013049a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:51.887153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd33dc-e800-4b56-9ab8-dfff8013049a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:51.888972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fd33dc-e800-4b56-9ab8-dfff8013049a service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:51.907995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1003/1991] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:52.654155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f22bb-2490-4ba8-80f8-5921562a6d47 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:52.654682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f22bb-2490-4ba8-80f8-5921562a6d47 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:52.654682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461f22bb-2490-4ba8-80f8-5921562a6d47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:52.686987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-461f22bb-2490-4ba8-80f8-5921562a6d47 None None] MFA Rules not processed for user `3eebc564cb2a4b26bdcf6cf3fd2cd744`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:52.688654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-461f22bb-2490-4ba8-80f8-5921562a6d47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:52.730127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:52.779543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 989/1992] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:43:52.784674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd33dc-e800-4b56-9ab8-dfff8013049a service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:52.794736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fd33dc-e800-4b56-9ab8-dfff8013049a service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:52.799812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9032263-81ec-4abc-96af-bcc9e267213f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:52.800424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9032263-81ec-4abc-96af-bcc9e267213f admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:52.800702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9032263-81ec-4abc-96af-bcc9e267213f admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:52.801014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9032263-81ec-4abc-96af-bcc9e267213f admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:52.801656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9032263-81ec-4abc-96af-bcc9e267213f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:52.803580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9032263-81ec-4abc-96af-bcc9e267213f admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:52.834190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1004/1993] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:52 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:43:52.838015 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f22bb-2490-4ba8-80f8-5921562a6d47 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:52.850050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461f22bb-2490-4ba8-80f8-5921562a6d47 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:52.909223 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c016f34-9efe-439c-a9e8-2b7dbd35f670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:52.909846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c016f34-9efe-439c-a9e8-2b7dbd35f670 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:52.910132 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c016f34-9efe-439c-a9e8-2b7dbd35f670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:52.910406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c016f34-9efe-439c-a9e8-2b7dbd35f670 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:52.911048 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c016f34-9efe-439c-a9e8-2b7dbd35f670 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:52.913005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c016f34-9efe-439c-a9e8-2b7dbd35f670 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:52.935508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 990/1994] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:52 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:52.940019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9032263-81ec-4abc-96af-bcc9e267213f admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:52.950156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9032263-81ec-4abc-96af-bcc9e267213f admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:52.955335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0444ba1-c32e-4e30-b634-f00ef2e24b84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:52.955928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0444ba1-c32e-4e30-b634-f00ef2e24b84 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:52.956200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0444ba1-c32e-4e30-b634-f00ef2e24b84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:52.956464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0444ba1-c32e-4e30-b634-f00ef2e24b84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:52.957135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0444ba1-c32e-4e30-b634-f00ef2e24b84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:52.958703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0444ba1-c32e-4e30-b634-f00ef2e24b84 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:52.969755 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1005/1995] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:52.974012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c016f34-9efe-439c-a9e8-2b7dbd35f670 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:52.984029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c016f34-9efe-439c-a9e8-2b7dbd35f670 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:52.989716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92e9ac-0cbf-4abc-ad2c-95c9d686512e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:52.990364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92e9ac-0cbf-4abc-ad2c-95c9d686512e admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:52.990639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92e9ac-0cbf-4abc-ad2c-95c9d686512e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:52.990911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa92e9ac-0cbf-4abc-ad2c-95c9d686512e admin admin] PATH_INFO: `/v3/projects/6d67b255035c4e10bbfb8827d4f615a8/users/b771c7a4bd474e948c343dc0f26e8895/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:53.006339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92e9ac-0cbf-4abc-ad2c-95c9d686512e admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d67b255035c4e10bbfb8827d4f615a8, user_id=b771c7a4bd474e948c343dc0f26e8895, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:53.009007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa92e9ac-0cbf-4abc-ad2c-95c9d686512e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:53.024660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 991/1996] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:52 2022] PUT /identity/v3/projects/6d67b255035c4e10bbfb8827d4f615a8/users/b771c7a4bd474e948c343dc0f26e8895/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:53.028700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0444ba1-c32e-4e30-b634-f00ef2e24b84 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:53.039686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0444ba1-c32e-4e30-b634-f00ef2e24b84 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:53.095313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684e8ed3-8c9c-4517-ae13-be87c777b654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:53.095909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e8ed3-8c9c-4517-ae13-be87c777b654 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:53.096203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e8ed3-8c9c-4517-ae13-be87c777b654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:53.096456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684e8ed3-8c9c-4517-ae13-be87c777b654 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:53.097107 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e8ed3-8c9c-4517-ae13-be87c777b654 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:53.098666 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684e8ed3-8c9c-4517-ae13-be87c777b654 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:53.108110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1006/1997] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:53.112243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92e9ac-0cbf-4abc-ad2c-95c9d686512e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:53.126535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa92e9ac-0cbf-4abc-ad2c-95c9d686512e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:53.131604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8001-0c40-4ce0-8029-63b6a8617d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:53.132263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8001-0c40-4ce0-8029-63b6a8617d4a admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:53.132539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8001-0c40-4ce0-8029-63b6a8617d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:53.132831 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c8001-0c40-4ce0-8029-63b6a8617d4a admin admin] PATH_INFO: `/v3/projects/6d67b255035c4e10bbfb8827d4f615a8/users/b771c7a4bd474e948c343dc0f26e8895/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:53.134827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c8001-0c40-4ce0-8029-63b6a8617d4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d67b255035c4e10bbfb8827d4f615a8, user_id=b771c7a4bd474e948c343dc0f26e8895, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:53.137474 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c8001-0c40-4ce0-8029-63b6a8617d4a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:53.151490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 992/1998] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:53 2022] PUT /identity/v3/projects/6d67b255035c4e10bbfb8827d4f615a8/users/b771c7a4bd474e948c343dc0f26e8895/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:53.810137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5b0d3-d0a3-4429-8c6c-a87a3154e36f None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:53.810655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5b0d3-d0a3-4429-8c6c-a87a3154e36f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:53.810761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d5b0d3-d0a3-4429-8c6c-a87a3154e36f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:53.841829 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-28d5b0d3-d0a3-4429-8c6c-a87a3154e36f None None] MFA Rules not processed for user `b771c7a4bd474e948c343dc0f26e8895`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:53.842592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-28d5b0d3-d0a3-4429-8c6c-a87a3154e36f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:53.889910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:53.947733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1007/1999] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:53.960457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8001-0c40-4ce0-8029-63b6a8617d4a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:53.962507 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5c3c8001-0c40-4ce0-8029-63b6a8617d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:54.050629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c8001-0c40-4ce0-8029-63b6a8617d4a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:54.054232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e23c8-024d-4f15-880c-28dc410040e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:54.054878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e23c8-024d-4f15-880c-28dc410040e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:54.055096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e23c8-024d-4f15-880c-28dc410040e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:54.055362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9e23c8-024d-4f15-880c-28dc410040e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:54.100783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:54.123538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e23c8-024d-4f15-880c-28dc410040e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:54.125854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9e23c8-024d-4f15-880c-28dc410040e5 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:54.135686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 993/2000] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:54.578655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5b0d3-d0a3-4429-8c6c-a87a3154e36f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:54.588717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d5b0d3-d0a3-4429-8c6c-a87a3154e36f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:54.592953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8bae4-c067-4b34-aa56-7788da2c3ddd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:54.594570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8bae4-c067-4b34-aa56-7788da2c3ddd service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:54.595053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8bae4-c067-4b34-aa56-7788da2c3ddd service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:54.595389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca8bae4-c067-4b34-aa56-7788da2c3ddd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:54.598124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-eca8bae4-c067-4b34-aa56-7788da2c3ddd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:54.666329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8bae4-c067-4b34-aa56-7788da2c3ddd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:54.667908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca8bae4-c067-4b34-aa56-7788da2c3ddd service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:54.678359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1008/2001] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:54 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:55.254998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e23c8-024d-4f15-880c-28dc410040e5 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:55.263349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9e23c8-024d-4f15-880c-28dc410040e5 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:55.267097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0881492-812d-4566-b182-f3e8aca0ba7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:55.267720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0881492-812d-4566-b182-f3e8aca0ba7d service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:55.267989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0881492-812d-4566-b182-f3e8aca0ba7d service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:55.268252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0881492-812d-4566-b182-f3e8aca0ba7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:55.332415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0881492-812d-4566-b182-f3e8aca0ba7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:55.333883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0881492-812d-4566-b182-f3e8aca0ba7d service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:55.343305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 994/2002] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:55.591175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8bae4-c067-4b34-aa56-7788da2c3ddd service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:55.600522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca8bae4-c067-4b34-aa56-7788da2c3ddd service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:55.605667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c11619-56be-4d5d-a848-1d8c5d6d98f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:55.606282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c11619-56be-4d5d-a848-1d8c5d6d98f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:55.606559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c11619-56be-4d5d-a848-1d8c5d6d98f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:55.606832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c11619-56be-4d5d-a848-1d8c5d6d98f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:55.619165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c11619-56be-4d5d-a848-1d8c5d6d98f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:55.620930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c11619-56be-4d5d-a848-1d8c5d6d98f2 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:55.641481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1009/2003] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:55 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:56.965176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0881492-812d-4566-b182-f3e8aca0ba7d service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:56.966756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c0881492-812d-4566-b182-f3e8aca0ba7d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:57.041111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0881492-812d-4566-b182-f3e8aca0ba7d service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:57.046058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:57.046729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:57.047890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:57.048608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] PATH_INFO: `/v3/users/e0b603469c0f4d4fa5a332aec545aca9` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:57.049721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b603469c0f4d4fa5a332aec545aca9)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:57.051829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:57.101867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b603469c0f4d4fa5a332aec545aca9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:57.102545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b603469c0f4d4fa5a332aec545aca9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:57.109241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b603469c0f4d4fa5a332aec545aca9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:57.117800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b603469c0f4d4fa5a332aec545aca9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:57.141499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:57.148475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b603469c0f4d4fa5a332aec545aca9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:57.155656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 995/2004] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:56 2022] DELETE /identity/v3/users/e0b603469c0f4d4fa5a332aec545aca9 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:57.160869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975a8a8-8e47-44cd-aeaf-4fed4e3cc2b8 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:57.161338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975a8a8-8e47-44cd-aeaf-4fed4e3cc2b8 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:57.161416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7975a8a8-8e47-44cd-aeaf-4fed4e3cc2b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:57.185131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7975a8a8-8e47-44cd-aeaf-4fed4e3cc2b8 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:57.186680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7975a8a8-8e47-44cd-aeaf-4fed4e3cc2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:57.248871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1010/2005] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:57.258698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:57.260177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:57.316458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44bc63b-f437-463e-bf1a-969c8e81f906 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:57.320053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97801941-5732-46e0-b059-0503eaa6f445 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:57.320651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97801941-5732-46e0-b059-0503eaa6f445 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:57.320951 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97801941-5732-46e0-b059-0503eaa6f445 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:57.321217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97801941-5732-46e0-b059-0503eaa6f445 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:57.330493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97801941-5732-46e0-b059-0503eaa6f445 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:57.331895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97801941-5732-46e0-b059-0503eaa6f445 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:57.341029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 996/2006] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:57.669430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975a8a8-8e47-44cd-aeaf-4fed4e3cc2b8 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:57.671120 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7975a8a8-8e47-44cd-aeaf-4fed4e3cc2b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:57.699813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7975a8a8-8e47-44cd-aeaf-4fed4e3cc2b8 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:57.705026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:57.705736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:57.705973 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:57.706240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] PATH_INFO: `/v3/projects/cc42dc9a609d441e848320a89013ad26` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:57.707542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc42dc9a609d441e848320a89013ad26)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:57.709692 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:57.762584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc42dc9a609d441e848320a89013ad26', 'request_id': 'req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:57.767030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:57.777690 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc42dc9a609d441e848320a89013ad26', 'request_id': 'req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:57.786149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc42dc9a609d441e848320a89013ad26', 'request_id': 'req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:57.798756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] The token cache is being invalidate because project cc42dc9a609d441e848320a89013ad26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:43:57.799048 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:57.799699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6222a3e8-b395-4640-ac9f-6ef1e7075cdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:57.806258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284579e2-aa07-4da4-babf-c404adfb3247 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:57.806548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284579e2-aa07-4da4-babf-c404adfb3247 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:57.806825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284579e2-aa07-4da4-babf-c404adfb3247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:57.808826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1011/2007] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:43:57 2022] DELETE /identity/v3/projects/cc42dc9a609d441e848320a89013ad26 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:57.829050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-284579e2-aa07-4da4-babf-c404adfb3247 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:57.830715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-284579e2-aa07-4da4-babf-c404adfb3247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:57.908449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 997/2008] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:57.916895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f25bce-db6d-4c52-9309-50f01cedf9cb None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:57.917175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f25bce-db6d-4c52-9309-50f01cedf9cb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:57.917444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f25bce-db6d-4c52-9309-50f01cedf9cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:57.938719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-a5f25bce-db6d-4c52-9309-50f01cedf9cb None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:57.940034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a5f25bce-db6d-4c52-9309-50f01cedf9cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:57.948033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1012/2009] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:57.952240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284579e2-aa07-4da4-babf-c404adfb3247 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:57.961866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284579e2-aa07-4da4-babf-c404adfb3247 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:57.966715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f912d5-9496-4bac-aec3-8428fa3d9cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:57.967334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f912d5-9496-4bac-aec3-8428fa3d9cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:57.967610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f912d5-9496-4bac-aec3-8428fa3d9cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:57.967881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f912d5-9496-4bac-aec3-8428fa3d9cf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:57.968520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f912d5-9496-4bac-aec3-8428fa3d9cf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:43:57.968886 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f912d5-9496-4bac-aec3-8428fa3d9cf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:57.970543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f912d5-9496-4bac-aec3-8428fa3d9cf8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:57.982799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 998/2010] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:43:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:43:57.991240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f25bce-db6d-4c52-9309-50f01cedf9cb None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:57.998329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f25bce-db6d-4c52-9309-50f01cedf9cb None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:58.003874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc627e0-b237-4025-8822-f6a1dc447197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:58.004500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc627e0-b237-4025-8822-f6a1dc447197 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:58.004783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc627e0-b237-4025-8822-f6a1dc447197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:58.005059 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc627e0-b237-4025-8822-f6a1dc447197 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:58.005733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc627e0-b237-4025-8822-f6a1dc447197 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:58.007624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc627e0-b237-4025-8822-f6a1dc447197 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:58.045759 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1013/2011] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:43:57 2022] POST /identity/v3/projects => generated 363 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:58.050027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f912d5-9496-4bac-aec3-8428fa3d9cf8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:58.060395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f912d5-9496-4bac-aec3-8428fa3d9cf8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:58.124366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06035c9f-dab1-4088-a1c3-23bd2851d740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:58.125796 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06035c9f-dab1-4088-a1c3-23bd2851d740 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:58.126307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06035c9f-dab1-4088-a1c3-23bd2851d740 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:58.126841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06035c9f-dab1-4088-a1c3-23bd2851d740 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:58.127569 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06035c9f-dab1-4088-a1c3-23bd2851d740 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:58.130238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06035c9f-dab1-4088-a1c3-23bd2851d740 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:58.159654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 999/2012] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:43:58 2022] POST /identity/v3/users => generated 282 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:43:58.163895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc627e0-b237-4025-8822-f6a1dc447197 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:58.179590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc627e0-b237-4025-8822-f6a1dc447197 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:58.184429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25f17c-33ca-4aca-994d-66ce975ef3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:58.185104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25f17c-33ca-4aca-994d-66ce975ef3fb admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:58.185381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25f17c-33ca-4aca-994d-66ce975ef3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:58.185650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b25f17c-33ca-4aca-994d-66ce975ef3fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:58.186304 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25f17c-33ca-4aca-994d-66ce975ef3fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:58.188598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b25f17c-33ca-4aca-994d-66ce975ef3fb admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:58.198490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1014/2013] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:58.202743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06035c9f-dab1-4088-a1c3-23bd2851d740 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:58.214180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06035c9f-dab1-4088-a1c3-23bd2851d740 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:58.219339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d602c800-6263-4df7-8ae2-a2962b0f6a64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:58.219987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c800-6263-4df7-8ae2-a2962b0f6a64 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:58.220226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c800-6263-4df7-8ae2-a2962b0f6a64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:58.220488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d602c800-6263-4df7-8ae2-a2962b0f6a64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:58.221184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602c800-6263-4df7-8ae2-a2962b0f6a64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:58.222763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d602c800-6263-4df7-8ae2-a2962b0f6a64 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:58.228440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:58.237360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1000/2014] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:43:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:43:58.241111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25f17c-33ca-4aca-994d-66ce975ef3fb admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:58.252278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b25f17c-33ca-4aca-994d-66ce975ef3fb admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:58.256984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894db2a4-8698-4d72-b3f6-b289b9dd3fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:58.257633 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894db2a4-8698-4d72-b3f6-b289b9dd3fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:58.257939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894db2a4-8698-4d72-b3f6-b289b9dd3fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:58.258213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894db2a4-8698-4d72-b3f6-b289b9dd3fe8 admin admin] PATH_INFO: `/v3/projects/7b93da1770e6470ea30f7d6661f7e96b/users/3fe8002c0b3545f291f4b0634e6f0302/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:58.274174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894db2a4-8698-4d72-b3f6-b289b9dd3fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b93da1770e6470ea30f7d6661f7e96b, user_id=3fe8002c0b3545f291f4b0634e6f0302, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:58.276960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894db2a4-8698-4d72-b3f6-b289b9dd3fe8 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:58.294277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1015/2015] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:43:58 2022] PUT /identity/v3/projects/7b93da1770e6470ea30f7d6661f7e96b/users/3fe8002c0b3545f291f4b0634e6f0302/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:43:58.308560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03693c29-9667-4ac1-9005-1ecac28dd125 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:58.308866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03693c29-9667-4ac1-9005-1ecac28dd125 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:58.309137 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03693c29-9667-4ac1-9005-1ecac28dd125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:58.337942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-03693c29-9667-4ac1-9005-1ecac28dd125 None None] MFA Rules not processed for user `3fe8002c0b3545f291f4b0634e6f0302`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:58.339170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-03693c29-9667-4ac1-9005-1ecac28dd125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:58.414256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1001/2016] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:58.426545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894db2a4-8698-4d72-b3f6-b289b9dd3fe8 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:58.428688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-894db2a4-8698-4d72-b3f6-b289b9dd3fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:58.491937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894db2a4-8698-4d72-b3f6-b289b9dd3fe8 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:58.495617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643a0cc4-479c-4038-b3c6-8fae31de7b7c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:58.496167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a0cc4-479c-4038-b3c6-8fae31de7b7c service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:58.496447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a0cc4-479c-4038-b3c6-8fae31de7b7c service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:58.496727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643a0cc4-479c-4038-b3c6-8fae31de7b7c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:58.504853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a0cc4-479c-4038-b3c6-8fae31de7b7c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:58.506263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643a0cc4-479c-4038-b3c6-8fae31de7b7c service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:58.514942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1016/2017] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:43:59.256936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03693c29-9667-4ac1-9005-1ecac28dd125 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:59.258394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-03693c29-9667-4ac1-9005-1ecac28dd125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:59.321806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03693c29-9667-4ac1-9005-1ecac28dd125 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:59.326265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:59.326881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:59.327152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:59.327418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] PATH_INFO: `/v3/users/9653434268634d11816643a9ac1a86f0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:59.338075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9653434268634d11816643a9ac1a86f0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:59.338075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:59.377500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9653434268634d11816643a9ac1a86f0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:59.378138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9653434268634d11816643a9ac1a86f0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:59.385989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9653434268634d11816643a9ac1a86f0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:59.392997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9653434268634d11816643a9ac1a86f0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:59.405286 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9653434268634d11816643a9ac1a86f0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:43:59.411452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1002/2018] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:43:59 2022] DELETE /identity/v3/users/9653434268634d11816643a9ac1a86f0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:43:59.416101 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da20c7-ec5a-4f3a-a8bf-9cbeac48d465 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:59.416569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da20c7-ec5a-4f3a-a8bf-9cbeac48d465 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:59.416721 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64da20c7-ec5a-4f3a-a8bf-9cbeac48d465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:59.435830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-64da20c7-ec5a-4f3a-a8bf-9cbeac48d465 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:43:59.437055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-64da20c7-ec5a-4f3a-a8bf-9cbeac48d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:59.494719 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1017/2019] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:43:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:43:59.505190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:43:59.506591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:43:59.517820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:43:59.572283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbd080e-c383-4949-8dd2-2ff928d981d2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:43:59.576423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768bcef-1aa6-49a5-8bf6-723c585d6769 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:43:59.577283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768bcef-1aa6-49a5-8bf6-723c585d6769 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:43:59.577558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768bcef-1aa6-49a5-8bf6-723c585d6769 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:43:59.577823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6768bcef-1aa6-49a5-8bf6-723c585d6769 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:43:59.587498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6768bcef-1aa6-49a5-8bf6-723c585d6769 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:43:59.588955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6768bcef-1aa6-49a5-8bf6-723c585d6769 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:43:59.598592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1003/2020] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:43:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:00.013664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da20c7-ec5a-4f3a-a8bf-9cbeac48d465 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:00.022954 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-64da20c7-ec5a-4f3a-a8bf-9cbeac48d465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:00.047309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64da20c7-ec5a-4f3a-a8bf-9cbeac48d465 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:00.055620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:00.057371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:00.057901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:00.058316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] PATH_INFO: `/v3/projects/aac81aafc28f4bd29a20fe59ed85ca3a` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:00.059986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aac81aafc28f4bd29a20fe59ed85ca3a)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:00.075364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:00.114335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:00.168568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aac81aafc28f4bd29a20fe59ed85ca3a', 'request_id': 'req-00733e25-0985-41b7-8277-a3f0b2b768c4'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:00.193549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aac81aafc28f4bd29a20fe59ed85ca3a', 'request_id': 'req-00733e25-0985-41b7-8277-a3f0b2b768c4'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:00.204531 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aac81aafc28f4bd29a20fe59ed85ca3a', 'request_id': 'req-00733e25-0985-41b7-8277-a3f0b2b768c4'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:00.233894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] The token cache is being invalidate because project aac81aafc28f4bd29a20fe59ed85ca3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:00.235190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:00.235963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:00.244159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1018/2021] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:00 2022] DELETE /identity/v3/projects/aac81aafc28f4bd29a20fe59ed85ca3a => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:00.245545 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768bcef-1aa6-49a5-8bf6-723c585d6769 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:00.262663 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6768bcef-1aa6-49a5-8bf6-723c585d6769 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:00.365632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:00.366459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:00.366843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:00.367212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] PATH_INFO: `/v3/users/d8bda835e87c40549746538fe48812ef` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:00.368531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8bda835e87c40549746538fe48812ef)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:00.371991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:00.465064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8bda835e87c40549746538fe48812ef'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:00.465848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8bda835e87c40549746538fe48812ef'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:00.475060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8bda835e87c40549746538fe48812ef'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:00.483076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8bda835e87c40549746538fe48812ef'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:00.499579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8bda835e87c40549746538fe48812ef'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:00.507266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1004/2022] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:00 2022] DELETE /identity/v3/users/d8bda835e87c40549746538fe48812ef => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:44:00.827950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:00.830148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:00.856924 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00733e25-0985-41b7-8277-a3f0b2b768c4 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:00.929443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:00.930600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:00.931106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:00.931797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] PATH_INFO: `/v3/projects/e34df43a48614f608e343e1250612372` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:00.933222 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] RBAC: Authorizing `identity:delete_project(project_id=e34df43a48614f608e343e1250612372)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:00.935953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:01.044659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e34df43a48614f608e343e1250612372', 'request_id': 'req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.069852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e34df43a48614f608e343e1250612372', 'request_id': 'req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.096645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e34df43a48614f608e343e1250612372', 'request_id': 'req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.124805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] The token cache is being invalidate because project e34df43a48614f608e343e1250612372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:01.125879 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.126933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.129669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1019/2023] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:00 2022] DELETE /identity/v3/projects/e34df43a48614f608e343e1250612372 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:01.133425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:01.135565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:01.170161 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2845545-e2b3-4a0d-8f65-57570c1bd7fe admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:01.239601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:01.280744 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e41a8-9608-4115-9147-3d8862a2dd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:01.281974 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e41a8-9608-4115-9147-3d8862a2dd0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:01.282108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e41a8-9608-4115-9147-3d8862a2dd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:01.282401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e41a8-9608-4115-9147-3d8862a2dd0e admin admin] PATH_INFO: `/v3/users/1824f98e9d6e4ed4938d1e14270c0b53` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:01.284427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e41a8-9608-4115-9147-3d8862a2dd0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=1824f98e9d6e4ed4938d1e14270c0b53)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:01.286426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e41a8-9608-4115-9147-3d8862a2dd0e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:01.350503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-310e41a8-9608-4115-9147-3d8862a2dd0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1824f98e9d6e4ed4938d1e14270c0b53'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.351263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-310e41a8-9608-4115-9147-3d8862a2dd0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1824f98e9d6e4ed4938d1e14270c0b53'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.362948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-310e41a8-9608-4115-9147-3d8862a2dd0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1824f98e9d6e4ed4938d1e14270c0b53'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.374116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-310e41a8-9608-4115-9147-3d8862a2dd0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1824f98e9d6e4ed4938d1e14270c0b53'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.393973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-310e41a8-9608-4115-9147-3d8862a2dd0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1824f98e9d6e4ed4938d1e14270c0b53'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.402250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1005/2024] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:01 2022] DELETE /identity/v3/users/1824f98e9d6e4ed4938d1e14270c0b53 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:01.483381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:01.493165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:01.515027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1de59d-99fe-40f7-8e5f-d6c9e4824fde admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:01.558834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:01.599526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:01.600547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:01.600979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:01.601372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] PATH_INFO: `/v3/users/c665c93c2b0941f3860d0e51cd48c63e` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:01.602711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c665c93c2b0941f3860d0e51cd48c63e)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:01.604632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:01.671076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c665c93c2b0941f3860d0e51cd48c63e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.680863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c665c93c2b0941f3860d0e51cd48c63e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.681755 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c665c93c2b0941f3860d0e51cd48c63e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.698882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c665c93c2b0941f3860d0e51cd48c63e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.713004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c665c93c2b0941f3860d0e51cd48c63e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.731440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1020/2025] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:01 2022] DELETE /identity/v3/users/c665c93c2b0941f3860d0e51cd48c63e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:01.736322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c1d1c-3066-41b3-b7f9-c29f7cec2017 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:01.736862 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c1d1c-3066-41b3-b7f9-c29f7cec2017 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:01.737326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1c1d1c-3066-41b3-b7f9-c29f7cec2017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:01.743042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:01.752499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:01.767296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0f1c1d1c-3066-41b3-b7f9-c29f7cec2017 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:01.768891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0f1c1d1c-3066-41b3-b7f9-c29f7cec2017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:01.836300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1006/2026] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:01.846037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a1e70-9558-40df-8128-6e6835e8c32e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:01.847521 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c1d1c-3066-41b3-b7f9-c29f7cec2017 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:01.848997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0f1c1d1c-3066-41b3-b7f9-c29f7cec2017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:01.852786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:01.853447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:01.853724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:01.854127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] PATH_INFO: `/v3/projects/83ea20527bf34830b7ac0646279d526a` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:01.855242 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] RBAC: Authorizing `identity:delete_project(project_id=83ea20527bf34830b7ac0646279d526a)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:01.857684 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:01.915521 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1c1d1c-3066-41b3-b7f9-c29f7cec2017 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:01.920077 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:01.922491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83ea20527bf34830b7ac0646279d526a', 'request_id': 'req-7d4c282d-bfa3-42fa-ae55-daf888d1933b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.926780 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:01.934361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83ea20527bf34830b7ac0646279d526a', 'request_id': 'req-7d4c282d-bfa3-42fa-ae55-daf888d1933b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.941697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83ea20527bf34830b7ac0646279d526a', 'request_id': 'req-7d4c282d-bfa3-42fa-ae55-daf888d1933b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.954706 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] The token cache is being invalidate because project 83ea20527bf34830b7ac0646279d526a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:01.955014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.955662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-7d4c282d-bfa3-42fa-ae55-daf888d1933b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:01.957403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1021/2027] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:01 2022] DELETE /identity/v3/projects/83ea20527bf34830b7ac0646279d526a => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:01.961778 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61b0c7-081b-47c6-9a23-e443c34efcfb None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:01.962053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61b0c7-081b-47c6-9a23-e443c34efcfb None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:01.962328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e61b0c7-081b-47c6-9a23-e443c34efcfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:01.974495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d2264-bf84-4d2c-a620-6df868b594bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:01.975234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d2264-bf84-4d2c-a620-6df868b594bb service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:01.975511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d2264-bf84-4d2c-a620-6df868b594bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:01.975806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d2264-bf84-4d2c-a620-6df868b594bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:01.977902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-945d2264-bf84-4d2c-a620-6df868b594bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:01.981737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7e61b0c7-081b-47c6-9a23-e443c34efcfb None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:01.982916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7e61b0c7-081b-47c6-9a23-e443c34efcfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:02.050031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1022/2028] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:02.051339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d2264-bf84-4d2c-a620-6df868b594bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:02.052980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d2264-bf84-4d2c-a620-6df868b594bb service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:02.057915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84f741-61a0-4d19-93f2-59b1919bbf0d None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.058197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84f741-61a0-4d19-93f2-59b1919bbf0d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.058472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84f741-61a0-4d19-93f2-59b1919bbf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.064538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1007/2029] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:44:02.079432 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7d84f741-61a0-4d19-93f2-59b1919bbf0d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:02.080834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7d84f741-61a0-4d19-93f2-59b1919bbf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:02.089284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1023/2030] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:02.093770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d2264-bf84-4d2c-a620-6df868b594bb service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:02.105570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d2264-bf84-4d2c-a620-6df868b594bb service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:02.111604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51a32d-8855-4db2-8788-323c67001cfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:02.112281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51a32d-8855-4db2-8788-323c67001cfa admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.112563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51a32d-8855-4db2-8788-323c67001cfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.112903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d51a32d-8855-4db2-8788-323c67001cfa admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.113574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51a32d-8855-4db2-8788-323c67001cfa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:02.113999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51a32d-8855-4db2-8788-323c67001cfa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:02.115670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d51a32d-8855-4db2-8788-323c67001cfa admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:02.128944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1008/2031] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:44:02.134186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84f741-61a0-4d19-93f2-59b1919bbf0d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:02.148578 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84f741-61a0-4d19-93f2-59b1919bbf0d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:02.154563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106cb27-ee3c-4680-9c45-2298f4f4d74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:02.155150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106cb27-ee3c-4680-9c45-2298f4f4d74b admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.155446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106cb27-ee3c-4680-9c45-2298f4f4d74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.155703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106cb27-ee3c-4680-9c45-2298f4f4d74b admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.156845 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106cb27-ee3c-4680-9c45-2298f4f4d74b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:02.158726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106cb27-ee3c-4680-9c45-2298f4f4d74b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:02.195998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1024/2032] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:02 2022] POST /identity/v3/projects => generated 389 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:02.200341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51a32d-8855-4db2-8788-323c67001cfa admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:02.211529 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d51a32d-8855-4db2-8788-323c67001cfa admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:02.282553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef93ca-0c66-4af7-9601-bc8d44090f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:02.283271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef93ca-0c66-4af7-9601-bc8d44090f25 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.284058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef93ca-0c66-4af7-9601-bc8d44090f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.284058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ef93ca-0c66-4af7-9601-bc8d44090f25 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.284505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef93ca-0c66-4af7-9601-bc8d44090f25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:02.287329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ef93ca-0c66-4af7-9601-bc8d44090f25 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:02.318392 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1009/2033] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:02 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:02.320091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106cb27-ee3c-4680-9c45-2298f4f4d74b admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:02.333635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106cb27-ee3c-4680-9c45-2298f4f4d74b admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:02.339026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bbba2-0164-4db7-8084-63a60bcc0281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:02.339852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bbba2-0164-4db7-8084-63a60bcc0281 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.341711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bbba2-0164-4db7-8084-63a60bcc0281 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.341994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5bbba2-0164-4db7-8084-63a60bcc0281 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.342888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bbba2-0164-4db7-8084-63a60bcc0281 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:02.344419 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5bbba2-0164-4db7-8084-63a60bcc0281 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:02.360294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1025/2034] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:02.362068 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef93ca-0c66-4af7-9601-bc8d44090f25 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:02.376560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ef93ca-0c66-4af7-9601-bc8d44090f25 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:02.392109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d749be-8591-41e9-a2a8-5762676c4ae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:02.396286 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d749be-8591-41e9-a2a8-5762676c4ae8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.396709 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d749be-8591-41e9-a2a8-5762676c4ae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.397234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d749be-8591-41e9-a2a8-5762676c4ae8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.398010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d749be-8591-41e9-a2a8-5762676c4ae8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:02.400347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d749be-8591-41e9-a2a8-5762676c4ae8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:02.416306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1010/2035] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:02 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:02.420517 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bbba2-0164-4db7-8084-63a60bcc0281 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:02.433498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5bbba2-0164-4db7-8084-63a60bcc0281 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:02.440385 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a517e9-3e6d-4b69-bab4-7afabe387e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:02.441132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a517e9-3e6d-4b69-bab4-7afabe387e16 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.441416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a517e9-3e6d-4b69-bab4-7afabe387e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.441692 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a517e9-3e6d-4b69-bab4-7afabe387e16 admin admin] PATH_INFO: `/v3/projects/d0c8e14e64d84b5eb4f3ece573fd00ae/users/aa3ee02fad1f4ebf901cf0c2febbc4d6/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.449316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d749be-8591-41e9-a2a8-5762676c4ae8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:02.451305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-75d749be-8591-41e9-a2a8-5762676c4ae8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:02.474402 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a517e9-3e6d-4b69-bab4-7afabe387e16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0c8e14e64d84b5eb4f3ece573fd00ae, user_id=aa3ee02fad1f4ebf901cf0c2febbc4d6, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:02.479356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a517e9-3e6d-4b69-bab4-7afabe387e16 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:02.502364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d749be-8591-41e9-a2a8-5762676c4ae8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:02.507369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1026/2036] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:02 2022] PUT /identity/v3/projects/d0c8e14e64d84b5eb4f3ece573fd00ae/users/aa3ee02fad1f4ebf901cf0c2febbc4d6/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:02.520791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c70977-79dd-4324-aa4c-edfdd35d9ee8 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.521120 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c70977-79dd-4324-aa4c-edfdd35d9ee8 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.521337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c70977-79dd-4324-aa4c-edfdd35d9ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.541647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:02.580936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-54c70977-79dd-4324-aa4c-edfdd35d9ee8 None None] MFA Rules not processed for user `aa3ee02fad1f4ebf901cf0c2febbc4d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:02.582262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-54c70977-79dd-4324-aa4c-edfdd35d9ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:02.605323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:02.606340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.606592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.606959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] PATH_INFO: `/v3/projects/c8e315b64f8b4a34ad9ce4f884617efb` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.608806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e315b64f8b4a34ad9ce4f884617efb)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:02.612424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:02.690625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e315b64f8b4a34ad9ce4f884617efb', 'request_id': 'req-d6b470eb-9e04-4e96-9179-046a5aae6976'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:02.703836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e315b64f8b4a34ad9ce4f884617efb', 'request_id': 'req-d6b470eb-9e04-4e96-9179-046a5aae6976'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:02.714539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e315b64f8b4a34ad9ce4f884617efb', 'request_id': 'req-d6b470eb-9e04-4e96-9179-046a5aae6976'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:02.733926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] The token cache is being invalidate because project c8e315b64f8b4a34ad9ce4f884617efb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:02.737138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:02.738275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d6b470eb-9e04-4e96-9179-046a5aae6976 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:02.740466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1011/2037] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:02 2022] DELETE /identity/v3/projects/c8e315b64f8b4a34ad9ce4f884617efb => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:02.747070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df00c1-4676-40d1-969e-8028c21f9762 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.747351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df00c1-4676-40d1-969e-8028c21f9762 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.747619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33df00c1-4676-40d1-969e-8028c21f9762 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.769489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1027/2038] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:02 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:02.777938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c70977-79dd-4324-aa4c-edfdd35d9ee8 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:02.779604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-54c70977-79dd-4324-aa4c-edfdd35d9ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:02.779817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-33df00c1-4676-40d1-969e-8028c21f9762 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:02.781854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-33df00c1-4676-40d1-969e-8028c21f9762 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:02.794533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:02.856840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c70977-79dd-4324-aa4c-edfdd35d9ee8 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:02.865846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1012/2039] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:02.873465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20c2d-14a8-4550-b533-565cc51d22eb None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.873759 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20c2d-14a8-4550-b533-565cc51d22eb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.875510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc20c2d-14a8-4550-b533-565cc51d22eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.903429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2bc20c2d-14a8-4550-b533-565cc51d22eb None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:02.905227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2bc20c2d-14a8-4550-b533-565cc51d22eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:02.914294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1013/2040] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:02.919445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc20c2d-14a8-4550-b533-565cc51d22eb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:02.929707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc20c2d-14a8-4550-b533-565cc51d22eb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:02.930521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b14eff-10e0-4973-9f8a-e8b394fb6ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:02.931214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b14eff-10e0-4973-9f8a-e8b394fb6ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.931395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b14eff-10e0-4973-9f8a-e8b394fb6ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.931654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b14eff-10e0-4973-9f8a-e8b394fb6ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.934756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d8086-8c29-4924-a21d-41f3517b9e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:02.935430 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d8086-8c29-4924-a21d-41f3517b9e8e admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.935773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d8086-8c29-4924-a21d-41f3517b9e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.936124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5d8086-8c29-4924-a21d-41f3517b9e8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.936896 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d8086-8c29-4924-a21d-41f3517b9e8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:02.937261 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d8086-8c29-4924-a21d-41f3517b9e8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:02.939841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5d8086-8c29-4924-a21d-41f3517b9e8e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:02.954626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1014/2041] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:44:02.960201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d8086-8c29-4924-a21d-41f3517b9e8e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:02.981088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5d8086-8c29-4924-a21d-41f3517b9e8e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:02.981088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136fde-c955-442e-ac84-2397331e9d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:02.981088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136fde-c955-442e-ac84-2397331e9d90 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:02.981088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136fde-c955-442e-ac84-2397331e9d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:02.981906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13136fde-c955-442e-ac84-2397331e9d90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:02.981906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13136fde-c955-442e-ac84-2397331e9d90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:02.981906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13136fde-c955-442e-ac84-2397331e9d90 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:03.005646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b14eff-10e0-4973-9f8a-e8b394fb6ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:03.007342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b14eff-10e0-4973-9f8a-e8b394fb6ff6 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:03.022046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1028/2042] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:02 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:03.022395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1015/2043] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:02 2022] POST /identity/v3/projects => generated 383 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:03.026620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b14eff-10e0-4973-9f8a-e8b394fb6ff6 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:03.039248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b14eff-10e0-4973-9f8a-e8b394fb6ff6 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:03.110647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656b032-2cea-4c95-9d6b-a3cd5880eb95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:03.111475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656b032-2cea-4c95-9d6b-a3cd5880eb95 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:03.111833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656b032-2cea-4c95-9d6b-a3cd5880eb95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:03.124911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b656b032-2cea-4c95-9d6b-a3cd5880eb95 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:03.124911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656b032-2cea-4c95-9d6b-a3cd5880eb95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:03.124911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b656b032-2cea-4c95-9d6b-a3cd5880eb95 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:03.150796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1029/2044] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:03 2022] POST /identity/v3/users => generated 292 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:03.152344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136fde-c955-442e-ac84-2397331e9d90 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:03.163577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13136fde-c955-442e-ac84-2397331e9d90 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:03.168468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c03c0-ad67-49c2-bcd5-d31b1c20e3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:03.169213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c03c0-ad67-49c2-bcd5-d31b1c20e3f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:03.169588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c03c0-ad67-49c2-bcd5-d31b1c20e3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:03.169954 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c03c0-ad67-49c2-bcd5-d31b1c20e3f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:03.170790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c03c0-ad67-49c2-bcd5-d31b1c20e3f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:03.172596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9c03c0-ad67-49c2-bcd5-d31b1c20e3f7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:03.178066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:03.186287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1016/2045] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:03.187739 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656b032-2cea-4c95-9d6b-a3cd5880eb95 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:03.199658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b656b032-2cea-4c95-9d6b-a3cd5880eb95 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:03.205649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32bd3aa-cc8d-484e-8da1-48a26f832e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:03.206311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bd3aa-cc8d-484e-8da1-48a26f832e96 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:03.206588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bd3aa-cc8d-484e-8da1-48a26f832e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:03.206855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32bd3aa-cc8d-484e-8da1-48a26f832e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:03.207489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bd3aa-cc8d-484e-8da1-48a26f832e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:03.209101 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32bd3aa-cc8d-484e-8da1-48a26f832e96 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:03.211580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:03.222556 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1030/2046] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:03.226720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c03c0-ad67-49c2-bcd5-d31b1c20e3f7 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:03.238470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c03c0-ad67-49c2-bcd5-d31b1c20e3f7 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:03.243495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e576c58-422e-45d7-af8d-105bbd4472b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:03.244145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e576c58-422e-45d7-af8d-105bbd4472b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:03.244457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e576c58-422e-45d7-af8d-105bbd4472b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:03.244792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e576c58-422e-45d7-af8d-105bbd4472b1 admin admin] PATH_INFO: `/v3/projects/12085c83b6ab43a196877fa8f5a30ce1/users/6f058265fa6842b5a8ab692b26090e84/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:03.267512 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e576c58-422e-45d7-af8d-105bbd4472b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12085c83b6ab43a196877fa8f5a30ce1, user_id=6f058265fa6842b5a8ab692b26090e84, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:03.270723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e576c58-422e-45d7-af8d-105bbd4472b1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:03.287458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1017/2047] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:03 2022] PUT /identity/v3/projects/12085c83b6ab43a196877fa8f5a30ce1/users/6f058265fa6842b5a8ab692b26090e84/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:03.299871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67278c51-94a6-4528-84c2-67e8298a513e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:03.300277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67278c51-94a6-4528-84c2-67e8298a513e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:03.301189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67278c51-94a6-4528-84c2-67e8298a513e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:03.354608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-67278c51-94a6-4528-84c2-67e8298a513e None None] MFA Rules not processed for user `6f058265fa6842b5a8ab692b26090e84`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:03.356714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-67278c51-94a6-4528-84c2-67e8298a513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:03.431445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1031/2048] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:03.443043 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e576c58-422e-45d7-af8d-105bbd4472b1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:03.498254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e576c58-422e-45d7-af8d-105bbd4472b1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:03.502648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae117b-e85d-4cd5-855e-5525a4a766c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:03.503251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae117b-e85d-4cd5-855e-5525a4a766c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:03.503524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae117b-e85d-4cd5-855e-5525a4a766c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:03.503788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ae117b-e85d-4cd5-855e-5525a4a766c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:03.511924 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae117b-e85d-4cd5-855e-5525a4a766c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:03.513393 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ae117b-e85d-4cd5-855e-5525a4a766c7 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:03.522910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1018/2049] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:07.826141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67278c51-94a6-4528-84c2-67e8298a513e None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:07.827702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-67278c51-94a6-4528-84c2-67e8298a513e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:07.901848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67278c51-94a6-4528-84c2-67e8298a513e None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:07.906335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992dee5c-60f3-4dba-9122-04ab8306e297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:07.906954 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dee5c-60f3-4dba-9122-04ab8306e297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:07.907226 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dee5c-60f3-4dba-9122-04ab8306e297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:07.907492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992dee5c-60f3-4dba-9122-04ab8306e297 admin admin] PATH_INFO: `/v3/users/da5264060ddd48afa8305c38a9bfa4c3` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:07.908587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992dee5c-60f3-4dba-9122-04ab8306e297 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da5264060ddd48afa8305c38a9bfa4c3)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:07.910505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992dee5c-60f3-4dba-9122-04ab8306e297 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:07.960588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-992dee5c-60f3-4dba-9122-04ab8306e297 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da5264060ddd48afa8305c38a9bfa4c3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:07.968270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-992dee5c-60f3-4dba-9122-04ab8306e297 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da5264060ddd48afa8305c38a9bfa4c3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:07.969986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-992dee5c-60f3-4dba-9122-04ab8306e297 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da5264060ddd48afa8305c38a9bfa4c3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:07.982315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-992dee5c-60f3-4dba-9122-04ab8306e297 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da5264060ddd48afa8305c38a9bfa4c3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:07.995926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-992dee5c-60f3-4dba-9122-04ab8306e297 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da5264060ddd48afa8305c38a9bfa4c3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:07.995926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1032/2050] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:07 2022] DELETE /identity/v3/users/da5264060ddd48afa8305c38a9bfa4c3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:08.164410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae117b-e85d-4cd5-855e-5525a4a766c7 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:08.176912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ae117b-e85d-4cd5-855e-5525a4a766c7 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:08.234115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:08.235347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:08.235534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:08.235796 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] PATH_INFO: `/v3/projects/2f01bb55623a4bf580008b75a9715796` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:08.236949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f01bb55623a4bf580008b75a9715796)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:08.238822 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:08.286773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f01bb55623a4bf580008b75a9715796', 'request_id': 'req-82b8eb89-d693-4345-8b9d-34f7eb54406d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:08.292834 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f01bb55623a4bf580008b75a9715796', 'request_id': 'req-82b8eb89-d693-4345-8b9d-34f7eb54406d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:08.310251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f01bb55623a4bf580008b75a9715796', 'request_id': 'req-82b8eb89-d693-4345-8b9d-34f7eb54406d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:08.323015 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] The token cache is being invalidate because project 2f01bb55623a4bf580008b75a9715796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:08.323336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:08.324307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-82b8eb89-d693-4345-8b9d-34f7eb54406d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:08.325857 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1019/2051] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:08 2022] DELETE /identity/v3/projects/2f01bb55623a4bf580008b75a9715796 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:08.331247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37aec-96b3-44b9-a758-bf543c7e20e4 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:08.331541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37aec-96b3-44b9-a758-bf543c7e20e4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:08.331817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d37aec-96b3-44b9-a758-bf543c7e20e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:08.345517 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:08.352915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-65d37aec-96b3-44b9-a758-bf543c7e20e4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:08.354663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-65d37aec-96b3-44b9-a758-bf543c7e20e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:08.413636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1033/2052] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:08.421300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de32885-5c06-4830-acac-87aa223412f6 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:08.421582 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de32885-5c06-4830-acac-87aa223412f6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:08.421856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de32885-5c06-4830-acac-87aa223412f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:08.442070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5de32885-5c06-4830-acac-87aa223412f6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:08.443257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5de32885-5c06-4830-acac-87aa223412f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:08.451041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1020/2053] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:08.456518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37aec-96b3-44b9-a758-bf543c7e20e4 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:08.466223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d37aec-96b3-44b9-a758-bf543c7e20e4 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:08.471257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18982a-80cf-48f4-994d-f65e1e0e0fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:08.471848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18982a-80cf-48f4-994d-f65e1e0e0fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:08.472125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18982a-80cf-48f4-994d-f65e1e0e0fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:08.472395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c18982a-80cf-48f4-994d-f65e1e0e0fdd admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:08.473089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18982a-80cf-48f4-994d-f65e1e0e0fdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:08.473386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18982a-80cf-48f4-994d-f65e1e0e0fdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:08.474955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c18982a-80cf-48f4-994d-f65e1e0e0fdd admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:08.487202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1034/2054] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:44:08.491418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de32885-5c06-4830-acac-87aa223412f6 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:08.495763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:08.502843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de32885-5c06-4830-acac-87aa223412f6 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:08.508464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c2fca-840e-4fde-950a-5b6d1a9e1091 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:08.509081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2fca-840e-4fde-950a-5b6d1a9e1091 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:08.509580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2fca-840e-4fde-950a-5b6d1a9e1091 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:08.509854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c2fca-840e-4fde-950a-5b6d1a9e1091 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:08.510467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c2fca-840e-4fde-950a-5b6d1a9e1091 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:08.512332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c2fca-840e-4fde-950a-5b6d1a9e1091 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:08.538980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1021/2055] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:08 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:08.542562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18982a-80cf-48f4-994d-f65e1e0e0fdd admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:08.554409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c18982a-80cf-48f4-994d-f65e1e0e0fdd admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:08.614920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab1576-8757-4d1e-ba2e-8b28a0360de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:08.615520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab1576-8757-4d1e-ba2e-8b28a0360de9 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:08.615795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab1576-8757-4d1e-ba2e-8b28a0360de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:08.616062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ab1576-8757-4d1e-ba2e-8b28a0360de9 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:08.616685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab1576-8757-4d1e-ba2e-8b28a0360de9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:08.618587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ab1576-8757-4d1e-ba2e-8b28a0360de9 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:08.644693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1035/2056] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:08 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:08.645934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c2fca-840e-4fde-950a-5b6d1a9e1091 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:08.656598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c2fca-840e-4fde-950a-5b6d1a9e1091 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:08.664185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366c04c-7417-4255-b799-4b01c94ef01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:08.664185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366c04c-7417-4255-b799-4b01c94ef01b admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:08.664185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366c04c-7417-4255-b799-4b01c94ef01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:08.664185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4366c04c-7417-4255-b799-4b01c94ef01b admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:08.664512 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366c04c-7417-4255-b799-4b01c94ef01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:08.667439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4366c04c-7417-4255-b799-4b01c94ef01b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:08.678899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1022/2057] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:08.682962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab1576-8757-4d1e-ba2e-8b28a0360de9 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:08.693651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ab1576-8757-4d1e-ba2e-8b28a0360de9 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:08.698423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c4efdd-677c-4a34-843a-b11f837dcab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:08.699166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4efdd-677c-4a34-843a-b11f837dcab3 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:08.699441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4efdd-677c-4a34-843a-b11f837dcab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:08.699705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c4efdd-677c-4a34-843a-b11f837dcab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:08.700447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4efdd-677c-4a34-843a-b11f837dcab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:08.702659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c4efdd-677c-4a34-843a-b11f837dcab3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:08.718559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1036/2058] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:08.720584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366c04c-7417-4255-b799-4b01c94ef01b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:08.730857 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4366c04c-7417-4255-b799-4b01c94ef01b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:08.735838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2265ee-2ade-4d02-b829-9afbaae6d16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:08.736494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2265ee-2ade-4d02-b829-9afbaae6d16b admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:08.736791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2265ee-2ade-4d02-b829-9afbaae6d16b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:08.737251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2265ee-2ade-4d02-b829-9afbaae6d16b admin admin] PATH_INFO: `/v3/projects/065af0e985d84fe78e170e6fab0b513b/users/1c5e5c3924214a0bba4e259013a282ce/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:08.754170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2265ee-2ade-4d02-b829-9afbaae6d16b admin admin] RBAC: Authorizing `identity:create_grant(project_id=065af0e985d84fe78e170e6fab0b513b, user_id=1c5e5c3924214a0bba4e259013a282ce, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:08.757020 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2265ee-2ade-4d02-b829-9afbaae6d16b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:08.780804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1023/2059] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:08 2022] PUT /identity/v3/projects/065af0e985d84fe78e170e6fab0b513b/users/1c5e5c3924214a0bba4e259013a282ce/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:08.784867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeca12a-c6de-4602-9555-44c70e2e09a9 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:08.785265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeca12a-c6de-4602-9555-44c70e2e09a9 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:08.785638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeca12a-c6de-4602-9555-44c70e2e09a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:08.818753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-4eeca12a-c6de-4602-9555-44c70e2e09a9 None None] MFA Rules not processed for user `1c5e5c3924214a0bba4e259013a282ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:08.820230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4eeca12a-c6de-4602-9555-44c70e2e09a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:08.927329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1037/2060] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:08.951148 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2265ee-2ade-4d02-b829-9afbaae6d16b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:08.953453 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5f2265ee-2ade-4d02-b829-9afbaae6d16b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:09.066232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2265ee-2ade-4d02-b829-9afbaae6d16b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:09.070990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b30946-edd4-4807-8ba5-06008f999381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:09.071695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b30946-edd4-4807-8ba5-06008f999381 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:09.072098 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b30946-edd4-4807-8ba5-06008f999381 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:09.072465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b30946-edd4-4807-8ba5-06008f999381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:09.086811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b30946-edd4-4807-8ba5-06008f999381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:09.088462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b30946-edd4-4807-8ba5-06008f999381 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:09.098694 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1024/2061] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:11.470844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeca12a-c6de-4602-9555-44c70e2e09a9 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:11.472335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4eeca12a-c6de-4602-9555-44c70e2e09a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:11.477754 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:11.541420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeca12a-c6de-4602-9555-44c70e2e09a9 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:11.545997 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:11.546629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:11.546904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:11.547206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] PATH_INFO: `/v3/users/6f058265fa6842b5a8ab692b26090e84` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:11.548386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f058265fa6842b5a8ab692b26090e84)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:11.550362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:11.597492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f058265fa6842b5a8ab692b26090e84'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:11.604952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f058265fa6842b5a8ab692b26090e84'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:11.605595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f058265fa6842b5a8ab692b26090e84'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:11.617469 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f058265fa6842b5a8ab692b26090e84'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:11.624181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f058265fa6842b5a8ab692b26090e84'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:11.630153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1038/2062] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:11 2022] DELETE /identity/v3/users/6f058265fa6842b5a8ab692b26090e84 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:11.634537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc16b8c-a781-40ca-8a1c-162a8c82e7ac None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:11.635042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc16b8c-a781-40ca-8a1c-162a8c82e7ac None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:11.635042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc16b8c-a781-40ca-8a1c-162a8c82e7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:11.654855 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4cc16b8c-a781-40ca-8a1c-162a8c82e7ac None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:11.655982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4cc16b8c-a781-40ca-8a1c-162a8c82e7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:11.719586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1025/2063] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:11.729472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:11.730897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:11.792914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c4794-f06f-4ece-b005-664b887522fa admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:11.796538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0e64b-44b5-4279-9b67-f46f79d9ecb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:11.797683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0e64b-44b5-4279-9b67-f46f79d9ecb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:11.797969 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0e64b-44b5-4279-9b67-f46f79d9ecb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:11.798241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0e64b-44b5-4279-9b67-f46f79d9ecb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:11.807644 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0e64b-44b5-4279-9b67-f46f79d9ecb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:11.809191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0e64b-44b5-4279-9b67-f46f79d9ecb7 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:11.819618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1039/2064] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:12.084405 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc16b8c-a781-40ca-8a1c-162a8c82e7ac None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:12.085878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4cc16b8c-a781-40ca-8a1c-162a8c82e7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:12.099275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc16b8c-a781-40ca-8a1c-162a8c82e7ac None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:12.112400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:12.113614 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:12.113906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:12.114083 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] PATH_INFO: `/v3/projects/12085c83b6ab43a196877fa8f5a30ce1` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:12.115695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=12085c83b6ab43a196877fa8f5a30ce1)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:12.117689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:12.162269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:12.166591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12085c83b6ab43a196877fa8f5a30ce1', 'request_id': 'req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:12.173661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12085c83b6ab43a196877fa8f5a30ce1', 'request_id': 'req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:12.180734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12085c83b6ab43a196877fa8f5a30ce1', 'request_id': 'req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:12.191754 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] The token cache is being invalidate because project 12085c83b6ab43a196877fa8f5a30ce1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:12.192041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:12.192668 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-08c01386-d59f-4845-ae0c-73b8b2f6ca6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:12.194186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1026/2065] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:12 2022] DELETE /identity/v3/projects/12085c83b6ab43a196877fa8f5a30ce1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:12.198959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82eae85-df4c-4e37-acc2-5cac62a1bb9f None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:12.199249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82eae85-df4c-4e37-acc2-5cac62a1bb9f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:12.199558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82eae85-df4c-4e37-acc2-5cac62a1bb9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:12.220545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e82eae85-df4c-4e37-acc2-5cac62a1bb9f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:12.221810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e82eae85-df4c-4e37-acc2-5cac62a1bb9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:12.241616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:12.283126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1040/2066] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:12.290782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24ea9-6840-457c-a400-0248d8ab68f1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:12.291065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24ea9-6840-457c-a400-0248d8ab68f1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:12.291336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f24ea9-6840-457c-a400-0248d8ab68f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:12.311173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-67f24ea9-6840-457c-a400-0248d8ab68f1 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:12.312366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-67f24ea9-6840-457c-a400-0248d8ab68f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:12.320828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1027/2067] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:12.324970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82eae85-df4c-4e37-acc2-5cac62a1bb9f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:12.334555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82eae85-df4c-4e37-acc2-5cac62a1bb9f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:12.339056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfc3f1-71b4-48a8-8300-af4993b17424 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:12.339640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfc3f1-71b4-48a8-8300-af4993b17424 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:12.339931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfc3f1-71b4-48a8-8300-af4993b17424 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:12.340183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dfc3f1-71b4-48a8-8300-af4993b17424 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:12.340851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfc3f1-71b4-48a8-8300-af4993b17424 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:12.341153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfc3f1-71b4-48a8-8300-af4993b17424 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:12.342713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55dfc3f1-71b4-48a8-8300-af4993b17424 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:12.354891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1041/2068] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:44:12.359223 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f24ea9-6840-457c-a400-0248d8ab68f1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:12.369636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f24ea9-6840-457c-a400-0248d8ab68f1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:12.374551 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630198d-228f-4ffb-b4ee-4cd303f16518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:12.376088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630198d-228f-4ffb-b4ee-4cd303f16518 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:12.376379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630198d-228f-4ffb-b4ee-4cd303f16518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:12.376655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b630198d-228f-4ffb-b4ee-4cd303f16518 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:12.377331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630198d-228f-4ffb-b4ee-4cd303f16518 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:12.379258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b630198d-228f-4ffb-b4ee-4cd303f16518 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:12.409371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1028/2069] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:12 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:12.413553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfc3f1-71b4-48a8-8300-af4993b17424 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:12.424081 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dfc3f1-71b4-48a8-8300-af4993b17424 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:12.482177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73e28a-ed4b-4d45-8346-be7bf59e845b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:12.482791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e28a-ed4b-4d45-8346-be7bf59e845b admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:12.483065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e28a-ed4b-4d45-8346-be7bf59e845b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:12.483333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73e28a-ed4b-4d45-8346-be7bf59e845b admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:12.483960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73e28a-ed4b-4d45-8346-be7bf59e845b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:12.486546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e73e28a-ed4b-4d45-8346-be7bf59e845b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:12.511735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1042/2070] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:12 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:12.513060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630198d-228f-4ffb-b4ee-4cd303f16518 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:12.523838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b630198d-228f-4ffb-b4ee-4cd303f16518 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:12.528731 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd968ec3-c45b-48b1-bbff-170f8130b31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:12.529359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd968ec3-c45b-48b1-bbff-170f8130b31b admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:12.529636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd968ec3-c45b-48b1-bbff-170f8130b31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:12.529900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd968ec3-c45b-48b1-bbff-170f8130b31b admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:12.530554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd968ec3-c45b-48b1-bbff-170f8130b31b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:12.532708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd968ec3-c45b-48b1-bbff-170f8130b31b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:12.542681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1029/2071] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:12.546602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73e28a-ed4b-4d45-8346-be7bf59e845b admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:12.559571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73e28a-ed4b-4d45-8346-be7bf59e845b admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:12.565540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53bdb671-29c7-44bc-a26a-72f1d1b91ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:12.566278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdb671-29c7-44bc-a26a-72f1d1b91ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:12.566660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdb671-29c7-44bc-a26a-72f1d1b91ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:12.567037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53bdb671-29c7-44bc-a26a-72f1d1b91ad4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:12.567786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bdb671-29c7-44bc-a26a-72f1d1b91ad4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:12.569547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53bdb671-29c7-44bc-a26a-72f1d1b91ad4 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:12.584214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1043/2072] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:12.588184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd968ec3-c45b-48b1-bbff-170f8130b31b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:12.599065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd968ec3-c45b-48b1-bbff-170f8130b31b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:12.604271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5ebea-277e-4cc6-bbce-9dc7e1cc1b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:12.604947 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ebea-277e-4cc6-bbce-9dc7e1cc1b20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:12.605220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ebea-277e-4cc6-bbce-9dc7e1cc1b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:12.605485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe5ebea-277e-4cc6-bbce-9dc7e1cc1b20 admin admin] PATH_INFO: `/v3/projects/1de7b194e88d4830aafa9d52a5f835b3/users/49cca06475b846b2b4f9b4a5efa4f02a/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:12.620872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5ebea-277e-4cc6-bbce-9dc7e1cc1b20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1de7b194e88d4830aafa9d52a5f835b3, user_id=49cca06475b846b2b4f9b4a5efa4f02a, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:12.623571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe5ebea-277e-4cc6-bbce-9dc7e1cc1b20 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:12.638190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1030/2073] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:12 2022] PUT /identity/v3/projects/1de7b194e88d4830aafa9d52a5f835b3/users/49cca06475b846b2b4f9b4a5efa4f02a/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:12.650479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e904b-9a09-44c2-a5bf-18c630252b22 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:12.650772 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e904b-9a09-44c2-a5bf-18c630252b22 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:12.651279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77e904b-9a09-44c2-a5bf-18c630252b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:12.679492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c77e904b-9a09-44c2-a5bf-18c630252b22 None None] MFA Rules not processed for user `49cca06475b846b2b4f9b4a5efa4f02a`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:12.680639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c77e904b-9a09-44c2-a5bf-18c630252b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:12.746020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1044/2074] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:12.757272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5ebea-277e-4cc6-bbce-9dc7e1cc1b20 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:12.758983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cbe5ebea-277e-4cc6-bbce-9dc7e1cc1b20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:12.816527 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe5ebea-277e-4cc6-bbce-9dc7e1cc1b20 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:12.820191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c78f4f-4cdb-4020-a3c6-269869b036b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:12.820806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c78f4f-4cdb-4020-a3c6-269869b036b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:12.821085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c78f4f-4cdb-4020-a3c6-269869b036b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:12.821350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c78f4f-4cdb-4020-a3c6-269869b036b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:12.830074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c78f4f-4cdb-4020-a3c6-269869b036b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:12.831487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c78f4f-4cdb-4020-a3c6-269869b036b2 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:12.840132 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1031/2075] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:16.881549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e904b-9a09-44c2-a5bf-18c630252b22 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:16.883016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c77e904b-9a09-44c2-a5bf-18c630252b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:16.905863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:16.950158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77e904b-9a09-44c2-a5bf-18c630252b22 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:16.955019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:16.955922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:16.956195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:16.956461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] PATH_INFO: `/v3/users/3eebc564cb2a4b26bdcf6cf3fd2cd744` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:16.957595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eebc564cb2a4b26bdcf6cf3fd2cd744)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:16.959473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:17.004334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eebc564cb2a4b26bdcf6cf3fd2cd744'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.012087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eebc564cb2a4b26bdcf6cf3fd2cd744'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.012701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eebc564cb2a4b26bdcf6cf3fd2cd744'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.027092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eebc564cb2a4b26bdcf6cf3fd2cd744'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.034368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eebc564cb2a4b26bdcf6cf3fd2cd744'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.040183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1045/2076] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:16 2022] DELETE /identity/v3/users/3eebc564cb2a4b26bdcf6cf3fd2cd744 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:17.391437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c78f4f-4cdb-4020-a3c6-269869b036b2 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:17.404064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c78f4f-4cdb-4020-a3c6-269869b036b2 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:17.472061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:17.472614 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:17.472922 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:17.473192 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] PATH_INFO: `/v3/projects/19855dafaf8542f0a58d8d0467503e88` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:17.474511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] RBAC: Authorizing `identity:delete_project(project_id=19855dafaf8542f0a58d8d0467503e88)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:17.476470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:17.489004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:17.541353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19855dafaf8542f0a58d8d0467503e88', 'request_id': 'req-63e02707-b27f-4bac-bf9a-c9580b61659f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.548344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19855dafaf8542f0a58d8d0467503e88', 'request_id': 'req-63e02707-b27f-4bac-bf9a-c9580b61659f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.556914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19855dafaf8542f0a58d8d0467503e88', 'request_id': 'req-63e02707-b27f-4bac-bf9a-c9580b61659f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.568126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] The token cache is being invalidate because project 19855dafaf8542f0a58d8d0467503e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:17.568437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.569161 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.570679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1032/2077] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:17 2022] DELETE /identity/v3/projects/19855dafaf8542f0a58d8d0467503e88 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:17.574763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:17.576241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:17.589610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78519065-592f-4f65-a444-5d387930fce8 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:17.645152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f623389e-6871-4f4e-8241-9ae8b362a1ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:17.645777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623389e-6871-4f4e-8241-9ae8b362a1ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:17.646056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623389e-6871-4f4e-8241-9ae8b362a1ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:17.646327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f623389e-6871-4f4e-8241-9ae8b362a1ca admin admin] PATH_INFO: `/v3/users/b771c7a4bd474e948c343dc0f26e8895` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:17.647449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623389e-6871-4f4e-8241-9ae8b362a1ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=b771c7a4bd474e948c343dc0f26e8895)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:17.649434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f623389e-6871-4f4e-8241-9ae8b362a1ca admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:17.691603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:17.697508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-f623389e-6871-4f4e-8241-9ae8b362a1ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b771c7a4bd474e948c343dc0f26e8895'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.705530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-f623389e-6871-4f4e-8241-9ae8b362a1ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b771c7a4bd474e948c343dc0f26e8895'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.706356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-f623389e-6871-4f4e-8241-9ae8b362a1ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b771c7a4bd474e948c343dc0f26e8895'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.719850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-f623389e-6871-4f4e-8241-9ae8b362a1ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b771c7a4bd474e948c343dc0f26e8895'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.729032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-f623389e-6871-4f4e-8241-9ae8b362a1ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b771c7a4bd474e948c343dc0f26e8895'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:17.737297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1046/2078] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:17 2022] DELETE /identity/v3/users/b771c7a4bd474e948c343dc0f26e8895 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:17.916604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:17.918146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:17.934053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e02707-b27f-4bac-bf9a-c9580b61659f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:17.991675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:17.992225 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:17.992499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:17.992779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] PATH_INFO: `/v3/projects/6d67b255035c4e10bbfb8827d4f615a8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:17.993890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d67b255035c4e10bbfb8827d4f615a8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:17.995765 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:18.043770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d67b255035c4e10bbfb8827d4f615a8', 'request_id': 'req-7a0616fa-98e2-4c01-8603-db01a94a3c89'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:18.051043 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d67b255035c4e10bbfb8827d4f615a8', 'request_id': 'req-7a0616fa-98e2-4c01-8603-db01a94a3c89'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:18.058539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d67b255035c4e10bbfb8827d4f615a8', 'request_id': 'req-7a0616fa-98e2-4c01-8603-db01a94a3c89'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:18.069640 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] The token cache is being invalidate because project 6d67b255035c4e10bbfb8827d4f615a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:18.069922 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:18.070542 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7a0616fa-98e2-4c01-8603-db01a94a3c89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:18.072040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1033/2079] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:17 2022] DELETE /identity/v3/projects/6d67b255035c4e10bbfb8827d4f615a8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:18.077014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672688e5-ad0e-4519-bced-26e8f313dd02 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.077489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672688e5-ad0e-4519-bced-26e8f313dd02 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.077600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672688e5-ad0e-4519-bced-26e8f313dd02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.097711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-672688e5-ad0e-4519-bced-26e8f313dd02 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:18.098953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-672688e5-ad0e-4519-bced-26e8f313dd02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:18.163645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1047/2080] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:18.173402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef76e8-408a-469c-96bb-a07dea9479aa None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.173766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef76e8-408a-469c-96bb-a07dea9479aa None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.174039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ef76e8-408a-469c-96bb-a07dea9479aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.194264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-86ef76e8-408a-469c-96bb-a07dea9479aa None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:18.195452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-86ef76e8-408a-469c-96bb-a07dea9479aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:18.203348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1034/2081] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:18.207573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672688e5-ad0e-4519-bced-26e8f313dd02 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:18.217085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672688e5-ad0e-4519-bced-26e8f313dd02 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:18.221617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e5910-d2d1-4460-968d-12150f1acaae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:18.222206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e5910-d2d1-4460-968d-12150f1acaae admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.222483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e5910-d2d1-4460-968d-12150f1acaae admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.222751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0e5910-d2d1-4460-968d-12150f1acaae admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.223383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e5910-d2d1-4460-968d-12150f1acaae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:18.223680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e5910-d2d1-4460-968d-12150f1acaae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:18.225292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0e5910-d2d1-4460-968d-12150f1acaae admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:18.237053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1048/2082] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:44:18.241146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef76e8-408a-469c-96bb-a07dea9479aa None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:18.251411 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ef76e8-408a-469c-96bb-a07dea9479aa None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:18.256270 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff141e7e-537b-49db-8bc3-7c72fa90d871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:18.256907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff141e7e-537b-49db-8bc3-7c72fa90d871 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.257187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff141e7e-537b-49db-8bc3-7c72fa90d871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.257463 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff141e7e-537b-49db-8bc3-7c72fa90d871 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.258091 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff141e7e-537b-49db-8bc3-7c72fa90d871 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:18.260043 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff141e7e-537b-49db-8bc3-7c72fa90d871 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:18.290155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1035/2083] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:18 2022] POST /identity/v3/projects => generated 359 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:18.294455 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e5910-d2d1-4460-968d-12150f1acaae admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:18.305302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0e5910-d2d1-4460-968d-12150f1acaae admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:18.360774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012c97f-adee-4a6e-a9e0-4f3833e74c4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:18.361373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012c97f-adee-4a6e-a9e0-4f3833e74c4c admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.361651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012c97f-adee-4a6e-a9e0-4f3833e74c4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.361925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4012c97f-adee-4a6e-a9e0-4f3833e74c4c admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.362543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012c97f-adee-4a6e-a9e0-4f3833e74c4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:18.364437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4012c97f-adee-4a6e-a9e0-4f3833e74c4c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:18.388809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1049/2084] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:18 2022] POST /identity/v3/users => generated 280 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:18.393232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff141e7e-537b-49db-8bc3-7c72fa90d871 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:18.398992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:18.405292 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff141e7e-537b-49db-8bc3-7c72fa90d871 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:18.411086 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb88486-6eb5-4692-b2c3-752bf4ed8bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:18.411677 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb88486-6eb5-4692-b2c3-752bf4ed8bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.411950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb88486-6eb5-4692-b2c3-752bf4ed8bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.412225 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb88486-6eb5-4692-b2c3-752bf4ed8bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.412916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb88486-6eb5-4692-b2c3-752bf4ed8bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:18.414540 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb88486-6eb5-4692-b2c3-752bf4ed8bb9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:18.425350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1036/2085] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:18.429321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012c97f-adee-4a6e-a9e0-4f3833e74c4c admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:18.439160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4012c97f-adee-4a6e-a9e0-4f3833e74c4c admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:18.444060 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbecb5-f58f-4fe0-849e-8419a2b1df0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:18.444710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbecb5-f58f-4fe0-849e-8419a2b1df0e admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.445045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbecb5-f58f-4fe0-849e-8419a2b1df0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.445283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbecb5-f58f-4fe0-849e-8419a2b1df0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.445946 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffbecb5-f58f-4fe0-849e-8419a2b1df0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:18.447539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffbecb5-f58f-4fe0-849e-8419a2b1df0e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:18.458092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1050/2086] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:18.462116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb88486-6eb5-4692-b2c3-752bf4ed8bb9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:18.472285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb88486-6eb5-4692-b2c3-752bf4ed8bb9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:18.476961 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98209cb8-3fca-4619-b941-f2b0688ff9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:18.477603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98209cb8-3fca-4619-b941-f2b0688ff9ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.477880 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98209cb8-3fca-4619-b941-f2b0688ff9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.478156 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98209cb8-3fca-4619-b941-f2b0688ff9ed admin admin] PATH_INFO: `/v3/projects/8792c04bfd984bc9b919eeb1d6ac64eb/users/a09a5b7b77cd448d9e42c67cad949bc2/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.493210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98209cb8-3fca-4619-b941-f2b0688ff9ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=8792c04bfd984bc9b919eeb1d6ac64eb, user_id=a09a5b7b77cd448d9e42c67cad949bc2, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:18.495780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98209cb8-3fca-4619-b941-f2b0688ff9ed admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:18.508110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1037/2087] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:18 2022] PUT /identity/v3/projects/8792c04bfd984bc9b919eeb1d6ac64eb/users/a09a5b7b77cd448d9e42c67cad949bc2/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:18.518745 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecad7e4-029d-4378-be44-2296025df5c1 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.519031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecad7e4-029d-4378-be44-2296025df5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.519310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecad7e4-029d-4378-be44-2296025df5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.555732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7ecad7e4-029d-4378-be44-2296025df5c1 None None] MFA Rules not processed for user `a09a5b7b77cd448d9e42c67cad949bc2`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:18.557149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7ecad7e4-029d-4378-be44-2296025df5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:18.569877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:18.632190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1051/2088] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:18.667842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98209cb8-3fca-4619-b941-f2b0688ff9ed admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:18.677583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-98209cb8-3fca-4619-b941-f2b0688ff9ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:18.738482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98209cb8-3fca-4619-b941-f2b0688ff9ed admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:18.742487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c869b0-eff4-4e31-b8de-b0c05208ba03 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:18.743092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c869b0-eff4-4e31-b8de-b0c05208ba03 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.743422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c869b0-eff4-4e31-b8de-b0c05208ba03 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.743641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c869b0-eff4-4e31-b8de-b0c05208ba03 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.752400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c869b0-eff4-4e31-b8de-b0c05208ba03 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:18.753964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c869b0-eff4-4e31-b8de-b0c05208ba03 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:18.762679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1038/2089] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:18 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:44:18.780108 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecad7e4-029d-4378-be44-2296025df5c1 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:18.853824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecad7e4-029d-4378-be44-2296025df5c1 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:18.858371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:18.859001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.859277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.860293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] PATH_INFO: `/v3/users/a09a5b7b77cd448d9e42c67cad949bc2` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.861921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a09a5b7b77cd448d9e42c67cad949bc2)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:18.863855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:18.915760 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a09a5b7b77cd448d9e42c67cad949bc2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:18.924789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a09a5b7b77cd448d9e42c67cad949bc2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:18.925919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a09a5b7b77cd448d9e42c67cad949bc2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:18.939935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a09a5b7b77cd448d9e42c67cad949bc2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:18.946825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a09a5b7b77cd448d9e42c67cad949bc2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:18.952578 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1052/2090] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:18 2022] DELETE /identity/v3/users/a09a5b7b77cd448d9e42c67cad949bc2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:18.957806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bcf51-eea3-499e-b075-8cf03704857f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:18.958165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bcf51-eea3-499e-b075-8cf03704857f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:18.958539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397bcf51-eea3-499e-b075-8cf03704857f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:18.978168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-397bcf51-eea3-499e-b075-8cf03704857f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:18.979433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-397bcf51-eea3-499e-b075-8cf03704857f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:19.045376 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1039/2091] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:19.055846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:19.057485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:19.097680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:19.130787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc91bb4-8043-4bc3-b95d-70f716a618f1 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:19.134849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa57fc6-1796-42c9-a619-b33f24bd07b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:19.135704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa57fc6-1796-42c9-a619-b33f24bd07b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:19.136082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa57fc6-1796-42c9-a619-b33f24bd07b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:19.136454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa57fc6-1796-42c9-a619-b33f24bd07b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:19.150092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa57fc6-1796-42c9-a619-b33f24bd07b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:19.151687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa57fc6-1796-42c9-a619-b33f24bd07b7 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:19.162894 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1053/2092] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:19.462390 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bcf51-eea3-499e-b075-8cf03704857f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:19.463974 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-397bcf51-eea3-499e-b075-8cf03704857f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:19.483756 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397bcf51-eea3-499e-b075-8cf03704857f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:19.489176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:19.489855 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:19.490201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:19.490382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] PATH_INFO: `/v3/projects/8792c04bfd984bc9b919eeb1d6ac64eb` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:19.491575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8792c04bfd984bc9b919eeb1d6ac64eb)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:19.493559 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:19.544024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8792c04bfd984bc9b919eeb1d6ac64eb', 'request_id': 'req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:19.554736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8792c04bfd984bc9b919eeb1d6ac64eb', 'request_id': 'req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:19.565406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8792c04bfd984bc9b919eeb1d6ac64eb', 'request_id': 'req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:19.571937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:19.581437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] The token cache is being invalidate because project 8792c04bfd984bc9b919eeb1d6ac64eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:19.582117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:19.583320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2e3fc2e3-eb50-4fbc-ad48-28229c77f2d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:19.585467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1040/2093] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:19 2022] DELETE /identity/v3/projects/8792c04bfd984bc9b919eeb1d6ac64eb => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:19.595246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc496-2bbc-4699-bc11-5d0cd09bf522 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:19.595748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc496-2bbc-4699-bc11-5d0cd09bf522 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:19.596159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897fc496-2bbc-4699-bc11-5d0cd09bf522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:19.622433 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-897fc496-2bbc-4699-bc11-5d0cd09bf522 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:19.624003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-897fc496-2bbc-4699-bc11-5d0cd09bf522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:19.768228 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1054/2094] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:19.782084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac7c7-c5ed-4f94-a7a8-7e9e644f79e7 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:19.782227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac7c7-c5ed-4f94-a7a8-7e9e644f79e7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:19.782418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485ac7c7-c5ed-4f94-a7a8-7e9e644f79e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:19.806383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-485ac7c7-c5ed-4f94-a7a8-7e9e644f79e7 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:19.807918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-485ac7c7-c5ed-4f94-a7a8-7e9e644f79e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:19.825327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fc496-2bbc-4699-bc11-5d0cd09bf522 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:19.827095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1041/2095] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:19.837345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897fc496-2bbc-4699-bc11-5d0cd09bf522 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:19.846779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4bdd2-3efc-45c8-aff7-4e21d7e84a65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:19.847670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4bdd2-3efc-45c8-aff7-4e21d7e84a65 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:19.848100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4bdd2-3efc-45c8-aff7-4e21d7e84a65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:19.848545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be4bdd2-3efc-45c8-aff7-4e21d7e84a65 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:19.849550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4bdd2-3efc-45c8-aff7-4e21d7e84a65 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:19.850232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4bdd2-3efc-45c8-aff7-4e21d7e84a65 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:19.852455 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be4bdd2-3efc-45c8-aff7-4e21d7e84a65 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:19.872959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1055/2096] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:44:19.876598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ac7c7-c5ed-4f94-a7a8-7e9e644f79e7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:19.889175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485ac7c7-c5ed-4f94-a7a8-7e9e644f79e7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:19.902382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aeb6f9-da5b-4b44-8a00-a573627f21bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:19.903005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aeb6f9-da5b-4b44-8a00-a573627f21bd admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:19.903277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aeb6f9-da5b-4b44-8a00-a573627f21bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:19.903554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4aeb6f9-da5b-4b44-8a00-a573627f21bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:19.904170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aeb6f9-da5b-4b44-8a00-a573627f21bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:19.906125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4aeb6f9-da5b-4b44-8a00-a573627f21bd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:19.942460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1042/2097] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:19 2022] POST /identity/v3/projects => generated 373 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:19.946995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4bdd2-3efc-45c8-aff7-4e21d7e84a65 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:19.958323 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be4bdd2-3efc-45c8-aff7-4e21d7e84a65 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:20.020912 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fa2ce-2e16-48ef-8f12-6bbfb033db0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:20.021525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fa2ce-2e16-48ef-8f12-6bbfb033db0c admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:20.022019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fa2ce-2e16-48ef-8f12-6bbfb033db0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:20.022408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665fa2ce-2e16-48ef-8f12-6bbfb033db0c admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:20.023062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fa2ce-2e16-48ef-8f12-6bbfb033db0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:20.024971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665fa2ce-2e16-48ef-8f12-6bbfb033db0c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:20.047911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1056/2098] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:19 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:20.052417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aeb6f9-da5b-4b44-8a00-a573627f21bd admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:20.063123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4aeb6f9-da5b-4b44-8a00-a573627f21bd admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:20.068090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3315571-6b4d-4683-8a2a-c44e81c0eade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:20.068619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315571-6b4d-4683-8a2a-c44e81c0eade admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:20.068932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315571-6b4d-4683-8a2a-c44e81c0eade admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:20.069204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3315571-6b4d-4683-8a2a-c44e81c0eade admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:20.069846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3315571-6b4d-4683-8a2a-c44e81c0eade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:20.071430 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3315571-6b4d-4683-8a2a-c44e81c0eade admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:20.082746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1043/2099] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:20.086441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fa2ce-2e16-48ef-8f12-6bbfb033db0c admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:20.096917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665fa2ce-2e16-48ef-8f12-6bbfb033db0c admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:20.102654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38b1182-5e03-44f0-9163-028b1b73610d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:20.103251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b1182-5e03-44f0-9163-028b1b73610d admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:20.103538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b1182-5e03-44f0-9163-028b1b73610d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:20.103809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38b1182-5e03-44f0-9163-028b1b73610d admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:20.104536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b1182-5e03-44f0-9163-028b1b73610d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:20.106163 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38b1182-5e03-44f0-9163-028b1b73610d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:20.122237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1057/2100] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:20.126904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3315571-6b4d-4683-8a2a-c44e81c0eade admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:20.138224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3315571-6b4d-4683-8a2a-c44e81c0eade admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:20.143216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ea40a-6d1d-4637-9bdd-f3a1a93f012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:20.143968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ea40a-6d1d-4637-9bdd-f3a1a93f012e admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:20.144247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ea40a-6d1d-4637-9bdd-f3a1a93f012e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:20.144790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ea40a-6d1d-4637-9bdd-f3a1a93f012e admin admin] PATH_INFO: `/v3/projects/a56a7316438141ddb79a7fabeefc162b/users/f4ffbcdd05f74a299c99188771117e48/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:20.164676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ea40a-6d1d-4637-9bdd-f3a1a93f012e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a56a7316438141ddb79a7fabeefc162b, user_id=f4ffbcdd05f74a299c99188771117e48, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:20.167469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ea40a-6d1d-4637-9bdd-f3a1a93f012e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:20.187821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1044/2101] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:20 2022] PUT /identity/v3/projects/a56a7316438141ddb79a7fabeefc162b/users/f4ffbcdd05f74a299c99188771117e48/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:20.193477 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b7b01-1631-4e64-82d6-27b626bfe604 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:20.193763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b7b01-1631-4e64-82d6-27b626bfe604 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:20.194043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81b7b01-1631-4e64-82d6-27b626bfe604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:20.228962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c81b7b01-1631-4e64-82d6-27b626bfe604 None None] MFA Rules not processed for user `f4ffbcdd05f74a299c99188771117e48`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:20.230182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c81b7b01-1631-4e64-82d6-27b626bfe604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:20.264470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:20.312667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1058/2102] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:20.330204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ea40a-6d1d-4637-9bdd-f3a1a93f012e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:20.331935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-064ea40a-6d1d-4637-9bdd-f3a1a93f012e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:20.399207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ea40a-6d1d-4637-9bdd-f3a1a93f012e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:20.402872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffdc27-4e53-4699-b071-9328b41fa54d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:20.403480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdc27-4e53-4699-b071-9328b41fa54d service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:20.403760 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdc27-4e53-4699-b071-9328b41fa54d service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:20.404031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdc27-4e53-4699-b071-9328b41fa54d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:20.413198 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffdc27-4e53-4699-b071-9328b41fa54d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:20.414675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffdc27-4e53-4699-b071-9328b41fa54d service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:20.425031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1045/2103] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:24.110041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b7b01-1631-4e64-82d6-27b626bfe604 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:24.172676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81b7b01-1631-4e64-82d6-27b626bfe604 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:24.177207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:24.177829 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:24.178103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:24.178371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] PATH_INFO: `/v3/users/f4ffbcdd05f74a299c99188771117e48` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:24.179476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4ffbcdd05f74a299c99188771117e48)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:24.181396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:24.226806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4ffbcdd05f74a299c99188771117e48'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.234199 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4ffbcdd05f74a299c99188771117e48'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.235074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4ffbcdd05f74a299c99188771117e48'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.247938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4ffbcdd05f74a299c99188771117e48'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.255119 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4ffbcdd05f74a299c99188771117e48'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.260951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1059/2104] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:24 2022] DELETE /identity/v3/users/f4ffbcdd05f74a299c99188771117e48 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:24.265523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c62a1-d0e0-44ce-a7a1-5174101e433d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:24.266631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c62a1-d0e0-44ce-a7a1-5174101e433d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:24.266631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8c62a1-d0e0-44ce-a7a1-5174101e433d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:24.286185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6d8c62a1-d0e0-44ce-a7a1-5174101e433d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:24.287373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6d8c62a1-d0e0-44ce-a7a1-5174101e433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:24.354401 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1046/2105] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:24.364891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:24.366379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:24.430515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736127c-7217-4b9d-a09c-a110a5870deb admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:24.434160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a61ad-39be-4d93-aa3e-0f026f6bb2c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:24.434760 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a61ad-39be-4d93-aa3e-0f026f6bb2c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:24.435032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a61ad-39be-4d93-aa3e-0f026f6bb2c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:24.435298 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6a61ad-39be-4d93-aa3e-0f026f6bb2c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:24.442589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:24.447823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a61ad-39be-4d93-aa3e-0f026f6bb2c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:24.449720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6a61ad-39be-4d93-aa3e-0f026f6bb2c7 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:24.460474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1060/2106] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:24.646622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c62a1-d0e0-44ce-a7a1-5174101e433d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:24.648251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6d8c62a1-d0e0-44ce-a7a1-5174101e433d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:24.662575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8c62a1-d0e0-44ce-a7a1-5174101e433d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:24.668659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162dcf50-5061-412f-a8d3-4aa562f9d5db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:24.669920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162dcf50-5061-412f-a8d3-4aa562f9d5db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:24.670345 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162dcf50-5061-412f-a8d3-4aa562f9d5db admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:24.670509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162dcf50-5061-412f-a8d3-4aa562f9d5db admin admin] PATH_INFO: `/v3/users/1c5e5c3924214a0bba4e259013a282ce` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:24.672111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162dcf50-5061-412f-a8d3-4aa562f9d5db admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5e5c3924214a0bba4e259013a282ce)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:24.674080 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162dcf50-5061-412f-a8d3-4aa562f9d5db admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:24.731372 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:24.753557 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a61ad-39be-4d93-aa3e-0f026f6bb2c7 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:24.755515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ff6a61ad-39be-4d93-aa3e-0f026f6bb2c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:24.772707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-162dcf50-5061-412f-a8d3-4aa562f9d5db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5e5c3924214a0bba4e259013a282ce'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.773313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6a61ad-39be-4d93-aa3e-0f026f6bb2c7 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:24.774265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-162dcf50-5061-412f-a8d3-4aa562f9d5db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5e5c3924214a0bba4e259013a282ce'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.785590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-162dcf50-5061-412f-a8d3-4aa562f9d5db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5e5c3924214a0bba4e259013a282ce'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.787323 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:24.789381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:24.789381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:24.789381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] PATH_INFO: `/v3/projects/a56a7316438141ddb79a7fabeefc162b` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:24.790698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] RBAC: Authorizing `identity:delete_project(project_id=a56a7316438141ddb79a7fabeefc162b)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:24.792669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:24.803124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-162dcf50-5061-412f-a8d3-4aa562f9d5db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5e5c3924214a0bba4e259013a282ce'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.825610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-162dcf50-5061-412f-a8d3-4aa562f9d5db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5e5c3924214a0bba4e259013a282ce'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.834369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1047/2107] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:24 2022] DELETE /identity/v3/users/1c5e5c3924214a0bba4e259013a282ce => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:24.839342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525445b-71ca-450b-af4d-64ba8a3cac60 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:24.839643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525445b-71ca-450b-af4d-64ba8a3cac60 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:24.839913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525445b-71ca-450b-af4d-64ba8a3cac60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:24.858950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a56a7316438141ddb79a7fabeefc162b', 'request_id': 'req-9a768df2-e86f-4494-ac60-5f6ea59bd82b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.862878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a525445b-71ca-450b-af4d-64ba8a3cac60 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:24.864296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a525445b-71ca-450b-af4d-64ba8a3cac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:24.868507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a56a7316438141ddb79a7fabeefc162b', 'request_id': 'req-9a768df2-e86f-4494-ac60-5f6ea59bd82b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.874073 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a56a7316438141ddb79a7fabeefc162b', 'request_id': 'req-9a768df2-e86f-4494-ac60-5f6ea59bd82b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.885984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] The token cache is being invalidate because project a56a7316438141ddb79a7fabeefc162b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:24.886269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.886907 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9a768df2-e86f-4494-ac60-5f6ea59bd82b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:24.888581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1061/2108] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:24 2022] DELETE /identity/v3/projects/a56a7316438141ddb79a7fabeefc162b => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:44:24.893829 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268880a8-1dd9-48b7-9aaa-e5721a90d79e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:24.894153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268880a8-1dd9-48b7-9aaa-e5721a90d79e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:24.894431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268880a8-1dd9-48b7-9aaa-e5721a90d79e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:24.917129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-268880a8-1dd9-48b7-9aaa-e5721a90d79e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:24.918348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-268880a8-1dd9-48b7-9aaa-e5721a90d79e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:24.934386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1048/2109] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:24.945119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525445b-71ca-450b-af4d-64ba8a3cac60 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:24.946899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a525445b-71ca-450b-af4d-64ba8a3cac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:24.982386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1062/2110] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:24.990470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a54db-c0ec-4b41-9467-463f39d74623 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:24.990757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a54db-c0ec-4b41-9467-463f39d74623 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:24.991037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92a54db-c0ec-4b41-9467-463f39d74623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.011405 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525445b-71ca-450b-af4d-64ba8a3cac60 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.011832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-b92a54db-c0ec-4b41-9467-463f39d74623 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:25.013087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b92a54db-c0ec-4b41-9467-463f39d74623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:25.015225 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7240be8-dd98-46a1-a832-928b53c76f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.016046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7240be8-dd98-46a1-a832-928b53c76f19 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.016547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7240be8-dd98-46a1-a832-928b53c76f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.017071 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7240be8-dd98-46a1-a832-928b53c76f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.022941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1063/2111] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:25.027241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a54db-c0ec-4b41-9467-463f39d74623 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.029022 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7240be8-dd98-46a1-a832-928b53c76f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.030735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7240be8-dd98-46a1-a832-928b53c76f19 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.037305 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92a54db-c0ec-4b41-9467-463f39d74623 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.040509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1049/2112] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:25.041909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509310ef-e778-4aea-9e4f-180e09dde053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.042509 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509310ef-e778-4aea-9e4f-180e09dde053 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.042786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509310ef-e778-4aea-9e4f-180e09dde053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.043054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509310ef-e778-4aea-9e4f-180e09dde053 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.043696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509310ef-e778-4aea-9e4f-180e09dde053 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:25.043997 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509310ef-e778-4aea-9e4f-180e09dde053 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.045620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509310ef-e778-4aea-9e4f-180e09dde053 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.058684 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1064/2113] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:44:25.063762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7240be8-dd98-46a1-a832-928b53c76f19 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.073142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7240be8-dd98-46a1-a832-928b53c76f19 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.077711 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0d5f5-c369-47f2-862b-232acf06323f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.078369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0d5f5-c369-47f2-862b-232acf06323f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.078647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0d5f5-c369-47f2-862b-232acf06323f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.078910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d0d5f5-c369-47f2-862b-232acf06323f admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.079524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0d5f5-c369-47f2-862b-232acf06323f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.081447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d0d5f5-c369-47f2-862b-232acf06323f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.117092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1050/2114] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:25 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:25.121624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509310ef-e778-4aea-9e4f-180e09dde053 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.133809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509310ef-e778-4aea-9e4f-180e09dde053 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.150118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0d5f5-c369-47f2-862b-232acf06323f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.152395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-90d0d5f5-c369-47f2-862b-232acf06323f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:25.190667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:25.200124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:25.206592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6f38-67e4-49c6-ad17-1720c8c73730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.207855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6f38-67e4-49c6-ad17-1720c8c73730 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.208701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6f38-67e4-49c6-ad17-1720c8c73730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.209291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bd6f38-67e4-49c6-ad17-1720c8c73730 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.210057 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd6f38-67e4-49c6-ad17-1720c8c73730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.213445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bd6f38-67e4-49c6-ad17-1720c8c73730 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.245232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1065/2115] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:25 2022] POST /identity/v3/users => generated 289 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:25.246430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6f38-67e4-49c6-ad17-1720c8c73730 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.254252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d0d5f5-c369-47f2-862b-232acf06323f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.257606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bd6f38-67e4-49c6-ad17-1720c8c73730 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.259417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.260492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.260492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.260698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] PATH_INFO: `/v3/users/aa3ee02fad1f4ebf901cf0c2febbc4d6` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.261843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa3ee02fad1f4ebf901cf0c2febbc4d6)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.263435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5a95-3ae8-46ed-8964-bee30c610538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.263863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.264160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5a95-3ae8-46ed-8964-bee30c610538 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.264562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5a95-3ae8-46ed-8964-bee30c610538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.264917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cf5a95-3ae8-46ed-8964-bee30c610538 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.266322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf5a95-3ae8-46ed-8964-bee30c610538 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.268412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cf5a95-3ae8-46ed-8964-bee30c610538 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.285902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1066/2116] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:25.290628 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5a95-3ae8-46ed-8964-bee30c610538 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.302454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cf5a95-3ae8-46ed-8964-bee30c610538 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.308125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a4392-2a37-44a3-880e-e83bd7fa6b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.308977 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a4392-2a37-44a3-880e-e83bd7fa6b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.309536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a4392-2a37-44a3-880e-e83bd7fa6b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.309826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297a4392-2a37-44a3-880e-e83bd7fa6b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.310470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a4392-2a37-44a3-880e-e83bd7fa6b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.312054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297a4392-2a37-44a3-880e-e83bd7fa6b49 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.323965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa3ee02fad1f4ebf901cf0c2febbc4d6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:25.325019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa3ee02fad1f4ebf901cf0c2febbc4d6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:25.327120 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1067/2117] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:25.332430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a4392-2a37-44a3-880e-e83bd7fa6b49 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.334817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa3ee02fad1f4ebf901cf0c2febbc4d6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:25.335247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-297a4392-2a37-44a3-880e-e83bd7fa6b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:25.345145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa3ee02fad1f4ebf901cf0c2febbc4d6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:25.355304 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297a4392-2a37-44a3-880e-e83bd7fa6b49 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.363523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aef0bc-9085-4600-8355-760563313bca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.364532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa3ee02fad1f4ebf901cf0c2febbc4d6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:25.364680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aef0bc-9085-4600-8355-760563313bca admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.364915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aef0bc-9085-4600-8355-760563313bca admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.365190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aef0bc-9085-4600-8355-760563313bca admin admin] PATH_INFO: `/v3/projects/cd8dd58cf5954b2da64bd9418f52e48b/users/c972e89c4e154ba3a1c27338e67adf5b/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.372843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1051/2118] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:25 2022] DELETE /identity/v3/users/aa3ee02fad1f4ebf901cf0c2febbc4d6 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:44:25.373912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.375944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:25.385620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aef0bc-9085-4600-8355-760563313bca admin admin] RBAC: Authorizing `identity:create_grant(project_id=cd8dd58cf5954b2da64bd9418f52e48b, user_id=c972e89c4e154ba3a1c27338e67adf5b, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.388132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aef0bc-9085-4600-8355-760563313bca admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.408807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1068/2119] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:25 2022] PUT /identity/v3/projects/cd8dd58cf5954b2da64bd9418f52e48b/users/c972e89c4e154ba3a1c27338e67adf5b/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:44:25.409320 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7ac92-dd81-452a-8522-f750f73084ff None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.409453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7ac92-dd81-452a-8522-f750f73084ff None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.409787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a7ac92-dd81-452a-8522-f750f73084ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.430238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-b4a7ac92-dd81-452a-8522-f750f73084ff None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:25.431540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b4a7ac92-dd81-452a-8522-f750f73084ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:25.459527 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb67705d-2fb1-48ae-9635-53adfbaa3c14 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.520136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:25.529142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1069/2120] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:25.530992 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b239f8-60db-4d34-918b-7d0c152fcebf None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.531619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b239f8-60db-4d34-918b-7d0c152fcebf None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.532391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b239f8-60db-4d34-918b-7d0c152fcebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.543574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.544245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.544657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.545016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] PATH_INFO: `/v3/projects/065af0e985d84fe78e170e6fab0b513b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.546472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=065af0e985d84fe78e170e6fab0b513b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.548681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.591133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-88b239f8-60db-4d34-918b-7d0c152fcebf None None] MFA Rules not processed for user `c972e89c4e154ba3a1c27338e67adf5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:25.592369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-88b239f8-60db-4d34-918b-7d0c152fcebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:25.625932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '065af0e985d84fe78e170e6fab0b513b', 'request_id': 'req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:25.632707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '065af0e985d84fe78e170e6fab0b513b', 'request_id': 'req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:25.641764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '065af0e985d84fe78e170e6fab0b513b', 'request_id': 'req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:25.652560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] The token cache is being invalidate because project 065af0e985d84fe78e170e6fab0b513b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:25.652953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:25.653660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:25.655242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1052/2121] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:25 2022] DELETE /identity/v3/projects/065af0e985d84fe78e170e6fab0b513b => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:25.656135 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.658077 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:25.672783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1070/2122] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:25 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:25.672783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177123fb-5f43-44cb-b2c9-7a38a924eda2 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.672783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177123fb-5f43-44cb-b2c9-7a38a924eda2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.672783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177123fb-5f43-44cb-b2c9-7a38a924eda2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.696094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-177123fb-5f43-44cb-b2c9-7a38a924eda2 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:25.697332 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-177123fb-5f43-44cb-b2c9-7a38a924eda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:25.718285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a414a85-4cbb-418b-b5f4-7c82caf26b6f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.723483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40e399e-dbef-42c2-82bc-7181698b64c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.724212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e399e-dbef-42c2-82bc-7181698b64c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.724485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e399e-dbef-42c2-82bc-7181698b64c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.724783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40e399e-dbef-42c2-82bc-7181698b64c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.726565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f40e399e-dbef-42c2-82bc-7181698b64c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:25.727487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:25.773607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1071/2123] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:25.774819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177123fb-5f43-44cb-b2c9-7a38a924eda2 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.776230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-177123fb-5f43-44cb-b2c9-7a38a924eda2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:25.789292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177123fb-5f43-44cb-b2c9-7a38a924eda2 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.793897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c92127-5611-4a53-b295-294a6de091d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.794487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c92127-5611-4a53-b295-294a6de091d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.794767 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c92127-5611-4a53-b295-294a6de091d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.795035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c92127-5611-4a53-b295-294a6de091d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.795652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c92127-5611-4a53-b295-294a6de091d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.797291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e399e-dbef-42c2-82bc-7181698b64c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.797575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c92127-5611-4a53-b295-294a6de091d8 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.798953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40e399e-dbef-42c2-82bc-7181698b64c7 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.809623 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1053/2124] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:25.811103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183b588-565a-470c-ba64-5e9a55779489 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.811403 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183b588-565a-470c-ba64-5e9a55779489 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.811680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e183b588-565a-470c-ba64-5e9a55779489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.832535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1072/2125] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:25 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:25.837607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c92127-5611-4a53-b295-294a6de091d8 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.837818 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e183b588-565a-470c-ba64-5e9a55779489 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:25.839184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e183b588-565a-470c-ba64-5e9a55779489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:25.877086 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:25.921178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1054/2126] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:25.925753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183b588-565a-470c-ba64-5e9a55779489 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.936397 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e183b588-565a-470c-ba64-5e9a55779489 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.943254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57533b6a-eb20-449a-a2d3-5ce999912f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.943482 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c92127-5611-4a53-b295-294a6de091d8 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.943884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57533b6a-eb20-449a-a2d3-5ce999912f0d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.944162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57533b6a-eb20-449a-a2d3-5ce999912f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.944447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57533b6a-eb20-449a-a2d3-5ce999912f0d admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.945118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57533b6a-eb20-449a-a2d3-5ce999912f0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:25.945421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57533b6a-eb20-449a-a2d3-5ce999912f0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.947816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57533b6a-eb20-449a-a2d3-5ce999912f0d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.949734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3777e7e-3c1c-47b5-a608-590777f7766d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.950742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3777e7e-3c1c-47b5-a608-590777f7766d admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.951161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3777e7e-3c1c-47b5-a608-590777f7766d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.951580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3777e7e-3c1c-47b5-a608-590777f7766d admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.952355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3777e7e-3c1c-47b5-a608-590777f7766d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.955088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3777e7e-3c1c-47b5-a608-590777f7766d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.961169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1055/2127] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:44:25.966238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57533b6a-eb20-449a-a2d3-5ce999912f0d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.979944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57533b6a-eb20-449a-a2d3-5ce999912f0d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:25.982240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1073/2128] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:25 2022] POST /identity/v3/users => generated 289 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:25.986348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efeb2a-7c42-4b93-85b8-794ec887dba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:25.986514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3777e7e-3c1c-47b5-a608-590777f7766d admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:25.987082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efeb2a-7c42-4b93-85b8-794ec887dba7 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:25.987483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efeb2a-7c42-4b93-85b8-794ec887dba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:25.987971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89efeb2a-7c42-4b93-85b8-794ec887dba7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:25.988807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efeb2a-7c42-4b93-85b8-794ec887dba7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:25.990708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89efeb2a-7c42-4b93-85b8-794ec887dba7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:25.997577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3777e7e-3c1c-47b5-a608-590777f7766d admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.002871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8aebf-2bfb-43ec-8900-4b67b8d347e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.004583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8aebf-2bfb-43ec-8900-4b67b8d347e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.004952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8aebf-2bfb-43ec-8900-4b67b8d347e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.005291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a8aebf-2bfb-43ec-8900-4b67b8d347e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.006004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8aebf-2bfb-43ec-8900-4b67b8d347e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.007773 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a8aebf-2bfb-43ec-8900-4b67b8d347e3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.025668 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1074/2129] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:25 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:26.029083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8aebf-2bfb-43ec-8900-4b67b8d347e3 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.032245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1056/2130] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:25 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:26.036119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efeb2a-7c42-4b93-85b8-794ec887dba7 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.040921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a8aebf-2bfb-43ec-8900-4b67b8d347e3 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.114085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a7637-0c7a-4137-a7ac-f24559cf3754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.114697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a7637-0c7a-4137-a7ac-f24559cf3754 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.115153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a7637-0c7a-4137-a7ac-f24559cf3754 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.115435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588a7637-0c7a-4137-a7ac-f24559cf3754 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.116082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a7637-0c7a-4137-a7ac-f24559cf3754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.117702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588a7637-0c7a-4137-a7ac-f24559cf3754 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.123318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89efeb2a-7c42-4b93-85b8-794ec887dba7 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.131678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1075/2131] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:26.133095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba74d63-b463-4284-b13a-4450778e8d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.133729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba74d63-b463-4284-b13a-4450778e8d52 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.134012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba74d63-b463-4284-b13a-4450778e8d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.134286 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba74d63-b463-4284-b13a-4450778e8d52 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.135207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba74d63-b463-4284-b13a-4450778e8d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.137197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba74d63-b463-4284-b13a-4450778e8d52 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.137738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a7637-0c7a-4137-a7ac-f24559cf3754 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.152428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588a7637-0c7a-4137-a7ac-f24559cf3754 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.161463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19bfff2-3db0-4098-91a9-3f7a956ac247 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.163049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bfff2-3db0-4098-91a9-3f7a956ac247 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.163230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bfff2-3db0-4098-91a9-3f7a956ac247 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.163580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19bfff2-3db0-4098-91a9-3f7a956ac247 admin admin] PATH_INFO: `/v3/projects/0c79723fd89b4ed0b2236b17f978ee25/users/94af4a692f9b4d479a97684d1af72e6e/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.167671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1057/2132] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:26 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:26.171496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba74d63-b463-4284-b13a-4450778e8d52 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.182436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba74d63-b463-4284-b13a-4450778e8d52 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.184632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bfff2-3db0-4098-91a9-3f7a956ac247 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c79723fd89b4ed0b2236b17f978ee25, user_id=94af4a692f9b4d479a97684d1af72e6e, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.187628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41151ee-9f16-4e1f-aee6-6ab0ea5c1ea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.187824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19bfff2-3db0-4098-91a9-3f7a956ac247 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.189151 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41151ee-9f16-4e1f-aee6-6ab0ea5c1ea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.189601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41151ee-9f16-4e1f-aee6-6ab0ea5c1ea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.190018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41151ee-9f16-4e1f-aee6-6ab0ea5c1ea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.190942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41151ee-9f16-4e1f-aee6-6ab0ea5c1ea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.193827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41151ee-9f16-4e1f-aee6-6ab0ea5c1ea1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.204446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:26.212351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1058/2133] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:26.212536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1076/2134] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:26 2022] PUT /identity/v3/projects/0c79723fd89b4ed0b2236b17f978ee25/users/94af4a692f9b4d479a97684d1af72e6e/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:44:26.215145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41151ee-9f16-4e1f-aee6-6ab0ea5c1ea1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.222415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da3185-34db-43d9-9ad5-f7b683bb380f None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.222769 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da3185-34db-43d9-9ad5-f7b683bb380f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.222963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99da3185-34db-43d9-9ad5-f7b683bb380f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.232791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41151ee-9f16-4e1f-aee6-6ab0ea5c1ea1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.255728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-99da3185-34db-43d9-9ad5-f7b683bb380f None None] MFA Rules not processed for user `94af4a692f9b4d479a97684d1af72e6e`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:26.257379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-99da3185-34db-43d9-9ad5-f7b683bb380f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:26.308609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9436c-e35a-4777-9bed-1d4b767a53d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.309382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9436c-e35a-4777-9bed-1d4b767a53d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.309751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9436c-e35a-4777-9bed-1d4b767a53d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.310118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9436c-e35a-4777-9bed-1d4b767a53d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.311188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9436c-e35a-4777-9bed-1d4b767a53d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.312862 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e9436c-e35a-4777-9bed-1d4b767a53d3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.323730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1059/2135] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:26.324509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9436c-e35a-4777-9bed-1d4b767a53d3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.329745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a6e9436c-e35a-4777-9bed-1d4b767a53d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:26.340082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1077/2136] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:26 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:26.341063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da3185-34db-43d9-9ad5-f7b683bb380f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.342553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9436c-e35a-4777-9bed-1d4b767a53d3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.348695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.348884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.349139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.349407 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] PATH_INFO: `/v3/projects/d0c8e14e64d84b5eb4f3ece573fd00ae` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.350506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0c8e14e64d84b5eb4f3ece573fd00ae)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.352380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.352963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99da3185-34db-43d9-9ad5-f7b683bb380f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.357832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8efd2-0dda-43fc-8d3e-ec983a909c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.358481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8efd2-0dda-43fc-8d3e-ec983a909c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.358757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8efd2-0dda-43fc-8d3e-ec983a909c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.359033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8efd2-0dda-43fc-8d3e-ec983a909c09 admin admin] PATH_INFO: `/v3/projects/ae11b961d07d4071ba3f2dfc84a0fec7/users/ba7380c17ca944e1bc550baee61078b2/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.361285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:26.375142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8efd2-0dda-43fc-8d3e-ec983a909c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae11b961d07d4071ba3f2dfc84a0fec7, user_id=ba7380c17ca944e1bc550baee61078b2, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.378282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8efd2-0dda-43fc-8d3e-ec983a909c09 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.402659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1078/2137] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:26 2022] PUT /identity/v3/projects/ae11b961d07d4071ba3f2dfc84a0fec7/users/ba7380c17ca944e1bc550baee61078b2/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:26.402659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8efd2-0dda-43fc-8d3e-ec983a909c09 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.422594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0c8e14e64d84b5eb4f3ece573fd00ae', 'request_id': 'req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:26.429503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0c8e14e64d84b5eb4f3ece573fd00ae', 'request_id': 'req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:26.438162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0c8e14e64d84b5eb4f3ece573fd00ae', 'request_id': 'req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:26.461019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] The token cache is being invalidate because project d0c8e14e64d84b5eb4f3ece573fd00ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:26.461463 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:26.462351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb91faee-d8ec-43c0-aae7-45e69e7f49cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:26.467727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1060/2138] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:26 2022] DELETE /identity/v3/projects/d0c8e14e64d84b5eb4f3ece573fd00ae => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:26.468864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839a6a2-c2f7-4a57-af0c-8bc36966b7c4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.469364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839a6a2-c2f7-4a57-af0c-8bc36966b7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.469845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839a6a2-c2f7-4a57-af0c-8bc36966b7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.479681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8efd2-0dda-43fc-8d3e-ec983a909c09 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.505349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e839a6a2-c2f7-4a57-af0c-8bc36966b7c4 None None] MFA Rules not processed for user `ba7380c17ca944e1bc550baee61078b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:26.506626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e839a6a2-c2f7-4a57-af0c-8bc36966b7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:26.567946 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3334fb9-7511-4a2c-991e-a3f11e506902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.568571 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3334fb9-7511-4a2c-991e-a3f11e506902 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.568895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3334fb9-7511-4a2c-991e-a3f11e506902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.569209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3334fb9-7511-4a2c-991e-a3f11e506902 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.569855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3334fb9-7511-4a2c-991e-a3f11e506902 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.576524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3334fb9-7511-4a2c-991e-a3f11e506902 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.607823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:26.626409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1079/2139] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:26 2022] POST /identity/v3/projects => generated 375 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:26.630168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3334fb9-7511-4a2c-991e-a3f11e506902 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.632277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a3334fb9-7511-4a2c-991e-a3f11e506902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:26.634596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1061/2140] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:26.641326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839a6a2-c2f7-4a57-af0c-8bc36966b7c4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.644899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e839a6a2-c2f7-4a57-af0c-8bc36966b7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:26.667910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839a6a2-c2f7-4a57-af0c-8bc36966b7c4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.729657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c1e78-94c9-475b-a586-30eac4ca082c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.730306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c1e78-94c9-475b-a586-30eac4ca082c admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.730577 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c1e78-94c9-475b-a586-30eac4ca082c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.730842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6c1e78-94c9-475b-a586-30eac4ca082c admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.731500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c1e78-94c9-475b-a586-30eac4ca082c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.733418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6c1e78-94c9-475b-a586-30eac4ca082c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.738219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3334fb9-7511-4a2c-991e-a3f11e506902 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.742881 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded88c4-c0f7-4773-b15c-f332dfc9ad2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.743483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded88c4-c0f7-4773-b15c-f332dfc9ad2a admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.743763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded88c4-c0f7-4773-b15c-f332dfc9ad2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.744033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ded88c4-c0f7-4773-b15c-f332dfc9ad2a admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.744649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded88c4-c0f7-4773-b15c-f332dfc9ad2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.746581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ded88c4-c0f7-4773-b15c-f332dfc9ad2a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.761373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1062/2141] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:26 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:26.762436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c1e78-94c9-475b-a586-30eac4ca082c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.764651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9b6c1e78-94c9-475b-a586-30eac4ca082c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:26.772629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1080/2142] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:26 2022] POST /identity/v3/users => generated 294 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:26.774939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded88c4-c0f7-4773-b15c-f332dfc9ad2a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.779031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:26.786259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ded88c4-c0f7-4773-b15c-f332dfc9ad2a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.839281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6c1e78-94c9-475b-a586-30eac4ca082c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.846824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.847454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.847738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.848031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] PATH_INFO: `/v3/users/49cca06475b846b2b4f9b4a5efa4f02a` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.849236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49cca06475b846b2b4f9b4a5efa4f02a)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.850667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c569a-59a6-4382-b86a-2f017f29f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.851539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c569a-59a6-4382-b86a-2f017f29f848 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.851819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c569a-59a6-4382-b86a-2f017f29f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.852157 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50c569a-59a6-4382-b86a-2f017f29f848 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.852899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c569a-59a6-4382-b86a-2f017f29f848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.854617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.855042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50c569a-59a6-4382-b86a-2f017f29f848 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.877422 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1081/2143] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:26 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:26.878316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c569a-59a6-4382-b86a-2f017f29f848 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.889550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50c569a-59a6-4382-b86a-2f017f29f848 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.895658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bc73d-48e0-4015-b886-d952360feecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.896260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bc73d-48e0-4015-b886-d952360feecc admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.896539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bc73d-48e0-4015-b886-d952360feecc admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.896843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807bc73d-48e0-4015-b886-d952360feecc admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.897487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bc73d-48e0-4015-b886-d952360feecc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.899093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807bc73d-48e0-4015-b886-d952360feecc admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.909193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49cca06475b846b2b4f9b4a5efa4f02a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:26.909905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49cca06475b846b2b4f9b4a5efa4f02a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:26.911088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1082/2144] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:26.911909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bc73d-48e0-4015-b886-d952360feecc admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.916052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-807bc73d-48e0-4015-b886-d952360feecc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:26.917789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49cca06475b846b2b4f9b4a5efa4f02a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:26.925776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49cca06475b846b2b4f9b4a5efa4f02a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:26.929757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807bc73d-48e0-4015-b886-d952360feecc admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:26.937465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd4c62a-2807-4d64-a7a5-f24de48af1a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:26.938119 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4c62a-2807-4d64-a7a5-f24de48af1a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.938395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4c62a-2807-4d64-a7a5-f24de48af1a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.938669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd4c62a-2807-4d64-a7a5-f24de48af1a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.939313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4c62a-2807-4d64-a7a5-f24de48af1a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:26.939489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49cca06475b846b2b4f9b4a5efa4f02a'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:26.941186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd4c62a-2807-4d64-a7a5-f24de48af1a9 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:26.945758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1063/2145] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:26 2022] DELETE /identity/v3/users/49cca06475b846b2b4f9b4a5efa4f02a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:26.946717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:26.948049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:26.953185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1083/2146] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:26.957497 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e40670-1706-49b6-bcc9-716ad25501e1 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:26.957785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e40670-1706-49b6-bcc9-716ad25501e1 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:26.958313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e40670-1706-49b6-bcc9-716ad25501e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:26.978675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d2e40670-1706-49b6-bcc9-716ad25501e1 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:26.979892 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d2e40670-1706-49b6-bcc9-716ad25501e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.014944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb151fc-f7ca-4dfe-8f8b-c8d8a1b0dcc6 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.020643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0a583-11d0-4322-adb2-702944dee7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.021362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0a583-11d0-4322-adb2-702944dee7cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.021634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0a583-11d0-4322-adb2-702944dee7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.021909 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf0a583-11d0-4322-adb2-702944dee7cf admin admin] PATH_INFO: `/v3/projects/f0448bb0d06047d5a4175eb84791f9ba/users/d64e87e0df9e41b8b5d62d0758ec3842/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.032843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:27.036980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0a583-11d0-4322-adb2-702944dee7cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0448bb0d06047d5a4175eb84791f9ba, user_id=d64e87e0df9e41b8b5d62d0758ec3842, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.039628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf0a583-11d0-4322-adb2-702944dee7cf admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.047897 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1084/2147] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:27.048993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e40670-1706-49b6-bcc9-716ad25501e1 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.054464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1064/2148] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:26 2022] PUT /identity/v3/projects/f0448bb0d06047d5a4175eb84791f9ba/users/d64e87e0df9e41b8b5d62d0758ec3842/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:27.057850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0a583-11d0-4322-adb2-702944dee7cf admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.058361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e40670-1706-49b6-bcc9-716ad25501e1 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.059483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dbf0a583-11d0-4322-adb2-702944dee7cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.082229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:27.120003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7066a-2719-4db5-b05a-994a37c2bf31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.120658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7066a-2719-4db5-b05a-994a37c2bf31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.120964 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7066a-2719-4db5-b05a-994a37c2bf31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.121233 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e7066a-2719-4db5-b05a-994a37c2bf31 admin admin] PATH_INFO: `/v3/projects/0a45a8e107d24325b7f0e0143a0aed5c/users/b93c47ee327544db9b1460c6b4913bad/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.131186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf0a583-11d0-4322-adb2-702944dee7cf admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.134867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7a2de-453d-4f82-ad1e-998d332d53fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.135493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7a2de-453d-4f82-ad1e-998d332d53fb service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.135772 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7a2de-453d-4f82-ad1e-998d332d53fb service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.136044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a7a2de-453d-4f82-ad1e-998d332d53fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.136207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7066a-2719-4db5-b05a-994a37c2bf31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a45a8e107d24325b7f0e0143a0aed5c, user_id=b93c47ee327544db9b1460c6b4913bad, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.138910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e7066a-2719-4db5-b05a-994a37c2bf31 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.151382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7a2de-453d-4f82-ad1e-998d332d53fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.152998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a7a2de-453d-4f82-ad1e-998d332d53fb service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.164925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1085/2149] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:27 2022] PUT /identity/v3/projects/0a45a8e107d24325b7f0e0143a0aed5c/users/b93c47ee327544db9b1460c6b4913bad/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:27.164925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7066a-2719-4db5-b05a-994a37c2bf31 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.165159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1065/2150] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:44:27.170522 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7a2de-453d-4f82-ad1e-998d332d53fb service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.183493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a7a2de-453d-4f82-ad1e-998d332d53fb service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.270521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e7066a-2719-4db5-b05a-994a37c2bf31 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.273009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d57a5-e76b-430c-a66c-8f96b40808fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.273288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d57a5-e76b-430c-a66c-8f96b40808fe admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.273795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d57a5-e76b-430c-a66c-8f96b40808fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.274312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7d57a5-e76b-430c-a66c-8f96b40808fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.275336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d57a5-e76b-430c-a66c-8f96b40808fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.277358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7d57a5-e76b-430c-a66c-8f96b40808fe admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.279489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09166332-ae8d-40ed-a618-c98e3cce3d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.280371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09166332-ae8d-40ed-a618-c98e3cce3d46 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.280855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09166332-ae8d-40ed-a618-c98e3cce3d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.281810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09166332-ae8d-40ed-a618-c98e3cce3d46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.282717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09166332-ae8d-40ed-a618-c98e3cce3d46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.284622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09166332-ae8d-40ed-a618-c98e3cce3d46 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.294202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1066/2151] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:27.299896 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d57a5-e76b-430c-a66c-8f96b40808fe admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.302503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1086/2152] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:27 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:27.307792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09166332-ae8d-40ed-a618-c98e3cce3d46 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.308095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7d57a5-e76b-430c-a66c-8f96b40808fe admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.312903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f433f28-2969-44ec-afd6-4b094bcea712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.313664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f433f28-2969-44ec-afd6-4b094bcea712 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.314011 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f433f28-2969-44ec-afd6-4b094bcea712 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.314346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f433f28-2969-44ec-afd6-4b094bcea712 admin admin] PATH_INFO: `/v3/projects/0a45a8e107d24325b7f0e0143a0aed5c/users/b93c47ee327544db9b1460c6b4913bad/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.317252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f433f28-2969-44ec-afd6-4b094bcea712 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a45a8e107d24325b7f0e0143a0aed5c, user_id=b93c47ee327544db9b1460c6b4913bad, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.320513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f433f28-2969-44ec-afd6-4b094bcea712 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.323065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09166332-ae8d-40ed-a618-c98e3cce3d46 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.330339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c38fd8-4019-4ae0-a55c-3c68a3efb0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.331356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c38fd8-4019-4ae0-a55c-3c68a3efb0ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.331797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c38fd8-4019-4ae0-a55c-3c68a3efb0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.332268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c38fd8-4019-4ae0-a55c-3c68a3efb0ef admin admin] PATH_INFO: `/v3/projects/f0448bb0d06047d5a4175eb84791f9ba/users/d64e87e0df9e41b8b5d62d0758ec3842/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.344308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1067/2153] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:27 2022] PUT /identity/v3/projects/0a45a8e107d24325b7f0e0143a0aed5c/users/b93c47ee327544db9b1460c6b4913bad/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:27.346917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c38fd8-4019-4ae0-a55c-3c68a3efb0ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0448bb0d06047d5a4175eb84791f9ba, user_id=d64e87e0df9e41b8b5d62d0758ec3842, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.350879 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8a513-bf03-4fdd-bd23-b9b4cd5fcc91 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.351233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8a513-bf03-4fdd-bd23-b9b4cd5fcc91 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.351573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df8a513-bf03-4fdd-bd23-b9b4cd5fcc91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.354573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c38fd8-4019-4ae0-a55c-3c68a3efb0ef admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.367621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1087/2154] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:27 2022] PUT /identity/v3/projects/f0448bb0d06047d5a4175eb84791f9ba/users/d64e87e0df9e41b8b5d62d0758ec3842/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:27.379093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471d856-53ec-4ee0-b827-67f168207d62 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.379639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471d856-53ec-4ee0-b827-67f168207d62 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.380067 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c471d856-53ec-4ee0-b827-67f168207d62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.401144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8df8a513-bf03-4fdd-bd23-b9b4cd5fcc91 None None] MFA Rules not processed for user `b93c47ee327544db9b1460c6b4913bad`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:27.402375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8df8a513-bf03-4fdd-bd23-b9b4cd5fcc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.414836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c471d856-53ec-4ee0-b827-67f168207d62 None None] MFA Rules not processed for user `d64e87e0df9e41b8b5d62d0758ec3842`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:27.415957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c471d856-53ec-4ee0-b827-67f168207d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.481048 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:27.495088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1068/2155] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:27 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:27.508487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8a513-bf03-4fdd-bd23-b9b4cd5fcc91 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.508877 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8df8a513-bf03-4fdd-bd23-b9b4cd5fcc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.517174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1088/2156] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:27.521735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471d856-53ec-4ee0-b827-67f168207d62 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.523068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c471d856-53ec-4ee0-b827-67f168207d62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.547826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:27.584500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df8a513-bf03-4fdd-bd23-b9b4cd5fcc91 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.588602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648892d2-093c-4784-a905-b70fdb3f0966 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.589250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648892d2-093c-4784-a905-b70fdb3f0966 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.589530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648892d2-093c-4784-a905-b70fdb3f0966 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.589800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648892d2-093c-4784-a905-b70fdb3f0966 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.599724 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648892d2-093c-4784-a905-b70fdb3f0966 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.601174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648892d2-093c-4784-a905-b70fdb3f0966 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.602747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c471d856-53ec-4ee0-b827-67f168207d62 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.607330 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.607952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.608223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.608488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] PATH_INFO: `/v3/projects/1de7b194e88d4830aafa9d52a5f835b3` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.609683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1de7b194e88d4830aafa9d52a5f835b3)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.611404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1069/2157] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:27 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:27.612194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648892d2-093c-4784-a905-b70fdb3f0966 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.612986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.613619 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-648892d2-093c-4784-a905-b70fdb3f0966 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.625870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648892d2-093c-4784-a905-b70fdb3f0966 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.630089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd11a0a-d651-499b-a539-f4e3be339c41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.630685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd11a0a-d651-499b-a539-f4e3be339c41 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.630958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd11a0a-d651-499b-a539-f4e3be339c41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.631226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd11a0a-d651-499b-a539-f4e3be339c41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.633001 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7fd11a0a-d651-499b-a539-f4e3be339c41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.675334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1de7b194e88d4830aafa9d52a5f835b3', 'request_id': 'req-408b1597-b65a-465f-a72a-9e455686725f'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:27.684443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1de7b194e88d4830aafa9d52a5f835b3', 'request_id': 'req-408b1597-b65a-465f-a72a-9e455686725f'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:27.691287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1de7b194e88d4830aafa9d52a5f835b3', 'request_id': 'req-408b1597-b65a-465f-a72a-9e455686725f'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:27.711967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] The token cache is being invalidate because project 1de7b194e88d4830aafa9d52a5f835b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:27.712421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:27.714547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:27.717106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1089/2158] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:27 2022] DELETE /identity/v3/projects/1de7b194e88d4830aafa9d52a5f835b3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:27.717232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.718756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.719091 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd11a0a-d651-499b-a539-f4e3be339c41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.720533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd11a0a-d651-499b-a539-f4e3be339c41 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.729722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1070/2159] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:27.732614 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4817b-1dd9-4d62-8846-37414c17a5e1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.732921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4817b-1dd9-4d62-8846-37414c17a5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.733290 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca4817b-1dd9-4d62-8846-37414c17a5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.755134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-7ca4817b-1dd9-4d62-8846-37414c17a5e1 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:27.756609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7ca4817b-1dd9-4d62-8846-37414c17a5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.783192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b1597-b65a-465f-a72a-9e455686725f admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.787695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c4f0b-17d5-4e09-adac-7d0bb136811b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.788512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c4f0b-17d5-4e09-adac-7d0bb136811b service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.788823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c4f0b-17d5-4e09-adac-7d0bb136811b service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.789105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3c4f0b-17d5-4e09-adac-7d0bb136811b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.790959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9e3c4f0b-17d5-4e09-adac-7d0bb136811b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.825926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1071/2160] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:27.832503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:27.834587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089d98f-afa3-47ca-b7c9-d25140867897 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.834977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089d98f-afa3-47ca-b7c9-d25140867897 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.834977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089d98f-afa3-47ca-b7c9-d25140867897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.855958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-9089d98f-afa3-47ca-b7c9-d25140867897 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:27.857317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9089d98f-afa3-47ca-b7c9-d25140867897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:27.867077 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1072/2161] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:27.871416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089d98f-afa3-47ca-b7c9-d25140867897 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.880167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c4f0b-17d5-4e09-adac-7d0bb136811b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.881486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9089d98f-afa3-47ca-b7c9-d25140867897 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.881844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3c4f0b-17d5-4e09-adac-7d0bb136811b service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.886211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01a44e-dc04-42a6-ad39-8cce7ac0e5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.887218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a44e-dc04-42a6-ad39-8cce7ac0e5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.887218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a44e-dc04-42a6-ad39-8cce7ac0e5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.887413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a01a44e-dc04-42a6-ad39-8cce7ac0e5e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.888231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01a44e-dc04-42a6-ad39-8cce7ac0e5e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:27.888231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01a44e-dc04-42a6-ad39-8cce7ac0e5e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.890015 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a01a44e-dc04-42a6-ad39-8cce7ac0e5e3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.892744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1090/2162] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:27.904159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1073/2163] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:44:27.909638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c4f0b-17d5-4e09-adac-7d0bb136811b service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.921724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3c4f0b-17d5-4e09-adac-7d0bb136811b service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:27.927113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d4623-960a-4fcb-995c-4edde5d3da5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:27.927655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d4623-960a-4fcb-995c-4edde5d3da5a admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:27.928055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d4623-960a-4fcb-995c-4edde5d3da5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:27.928540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4d4623-960a-4fcb-995c-4edde5d3da5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:27.929367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d4623-960a-4fcb-995c-4edde5d3da5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:27.931536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4d4623-960a-4fcb-995c-4edde5d3da5a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:27.963370 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1091/2164] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:27 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:27.967661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01a44e-dc04-42a6-ad39-8cce7ac0e5e3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:27.987295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a01a44e-dc04-42a6-ad39-8cce7ac0e5e3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:28.048915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:28.080516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198ba13-3402-4cce-9623-bcea2496fec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:28.081048 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198ba13-3402-4cce-9623-bcea2496fec9 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:28.081333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198ba13-3402-4cce-9623-bcea2496fec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:28.081610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e198ba13-3402-4cce-9623-bcea2496fec9 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:28.082263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198ba13-3402-4cce-9623-bcea2496fec9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:28.085419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e198ba13-3402-4cce-9623-bcea2496fec9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:28.125412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1074/2165] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:27 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:28.126796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d4623-960a-4fcb-995c-4edde5d3da5a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:28.138235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4d4623-960a-4fcb-995c-4edde5d3da5a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:28.144365 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cffed-033b-400b-8f24-7ae58e248841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:28.145284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cffed-033b-400b-8f24-7ae58e248841 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:28.145558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cffed-033b-400b-8f24-7ae58e248841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:28.145831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cffed-033b-400b-8f24-7ae58e248841 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:28.146506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cffed-033b-400b-8f24-7ae58e248841 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:28.148178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cffed-033b-400b-8f24-7ae58e248841 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:28.161162 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1092/2166] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:28.165380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198ba13-3402-4cce-9623-bcea2496fec9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:28.176560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e198ba13-3402-4cce-9623-bcea2496fec9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:28.182568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eded8b1a-9c14-48ba-ad84-c3b288327843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:28.183106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded8b1a-9c14-48ba-ad84-c3b288327843 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:28.183379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded8b1a-9c14-48ba-ad84-c3b288327843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:28.183642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eded8b1a-9c14-48ba-ad84-c3b288327843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:28.184301 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded8b1a-9c14-48ba-ad84-c3b288327843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:28.185939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eded8b1a-9c14-48ba-ad84-c3b288327843 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:28.198257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1075/2167] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:28 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:28.206260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cffed-033b-400b-8f24-7ae58e248841 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:28.219457 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cffed-033b-400b-8f24-7ae58e248841 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:28.226008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6672ede-7172-4547-81cd-2e096478617a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:28.226855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6672ede-7172-4547-81cd-2e096478617a admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:28.227197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6672ede-7172-4547-81cd-2e096478617a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:28.227534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6672ede-7172-4547-81cd-2e096478617a admin admin] PATH_INFO: `/v3/projects/3ebbea5186774c59942358c1fc284fa7/users/81d8036279ef4384bdbfa8c37e46a1d6/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:28.244561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6672ede-7172-4547-81cd-2e096478617a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ebbea5186774c59942358c1fc284fa7, user_id=81d8036279ef4384bdbfa8c37e46a1d6, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:28.247364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6672ede-7172-4547-81cd-2e096478617a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:28.263229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1093/2168] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:28 2022] PUT /identity/v3/projects/3ebbea5186774c59942358c1fc284fa7/users/81d8036279ef4384bdbfa8c37e46a1d6/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:28.269265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6d13f-3b9b-4162-948a-0f9ffd5c046e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:28.269956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6d13f-3b9b-4162-948a-0f9ffd5c046e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:28.270243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b6d13f-3b9b-4162-948a-0f9ffd5c046e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:28.290828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c2b6d13f-3b9b-4162-948a-0f9ffd5c046e None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:28.292264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c2b6d13f-3b9b-4162-948a-0f9ffd5c046e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:28.354798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1076/2169] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:28.364496 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6672ede-7172-4547-81cd-2e096478617a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:28.366219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a6672ede-7172-4547-81cd-2e096478617a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:28.435637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6672ede-7172-4547-81cd-2e096478617a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:28.440280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3764d-fff7-4f90-acff-82d9269885e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:28.440947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3764d-fff7-4f90-acff-82d9269885e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:28.441573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3764d-fff7-4f90-acff-82d9269885e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:28.441573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe3764d-fff7-4f90-acff-82d9269885e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:28.450976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3764d-fff7-4f90-acff-82d9269885e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:28.452392 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe3764d-fff7-4f90-acff-82d9269885e2 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:28.462956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1094/2170] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:29.120560 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30393d-903f-4ae1-9d00-de7526f2b3f1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:29.120910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30393d-903f-4ae1-9d00-de7526f2b3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:29.121141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30393d-903f-4ae1-9d00-de7526f2b3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:29.164112 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-fa30393d-903f-4ae1-9d00-de7526f2b3f1 None None] MFA Rules not processed for user `81d8036279ef4384bdbfa8c37e46a1d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:29.177042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fa30393d-903f-4ae1-9d00-de7526f2b3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:29.248859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1077/2171] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:29 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:29.261458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3764d-fff7-4f90-acff-82d9269885e2 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:29.325498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe3764d-fff7-4f90-acff-82d9269885e2 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:29.330475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa915873-253d-493a-9012-192ba5d882bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:29.331159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa915873-253d-493a-9012-192ba5d882bb service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:29.331507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa915873-253d-493a-9012-192ba5d882bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:29.332192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa915873-253d-493a-9012-192ba5d882bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:29.341923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa915873-253d-493a-9012-192ba5d882bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:29.343421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa915873-253d-493a-9012-192ba5d882bb service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:29.368703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1095/2172] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:29 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:29.432912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30393d-903f-4ae1-9d00-de7526f2b3f1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:29.434454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fa30393d-903f-4ae1-9d00-de7526f2b3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:29.448173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30393d-903f-4ae1-9d00-de7526f2b3f1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:29.452003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e852819-7dd0-4cd6-ad68-2b2fe7999c64 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:29.452605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e852819-7dd0-4cd6-ad68-2b2fe7999c64 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:29.452949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e852819-7dd0-4cd6-ad68-2b2fe7999c64 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:29.453866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e852819-7dd0-4cd6-ad68-2b2fe7999c64 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:29.455678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9e852819-7dd0-4cd6-ad68-2b2fe7999c64 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:29.513475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:29.518784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e852819-7dd0-4cd6-ad68-2b2fe7999c64 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:29.521694 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e852819-7dd0-4cd6-ad68-2b2fe7999c64 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:29.532156 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1078/2173] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:29 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:30.450409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa915873-253d-493a-9012-192ba5d882bb service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:30.460013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa915873-253d-493a-9012-192ba5d882bb service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:30.463626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41115572-53f9-4dd0-a660-c08e69129453 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:30.464215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41115572-53f9-4dd0-a660-c08e69129453 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:30.464496 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41115572-53f9-4dd0-a660-c08e69129453 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:30.464786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41115572-53f9-4dd0-a660-c08e69129453 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:30.466503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-41115572-53f9-4dd0-a660-c08e69129453 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:30.483878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:30.534596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41115572-53f9-4dd0-a660-c08e69129453 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:30.536056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41115572-53f9-4dd0-a660-c08e69129453 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:30.546503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1096/2174] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:30.855740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee3767-d2f3-4e8a-9b0c-9fa6276e3aba None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:30.856196 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee3767-d2f3-4e8a-9b0c-9fa6276e3aba None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:30.856385 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ee3767-d2f3-4e8a-9b0c-9fa6276e3aba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:30.883451 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f2ee3767-d2f3-4e8a-9b0c-9fa6276e3aba None None] MFA Rules not processed for user `81d8036279ef4384bdbfa8c37e46a1d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:30.885203 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f2ee3767-d2f3-4e8a-9b0c-9fa6276e3aba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:30.892826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1079/2175] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:30 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:31.924356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41115572-53f9-4dd0-a660-c08e69129453 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:31.926877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-41115572-53f9-4dd0-a660-c08e69129453 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:31.945630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41115572-53f9-4dd0-a660-c08e69129453 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:31.952379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:31.953918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:31.954218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:31.954648 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] PATH_INFO: `/v3/users/ba7380c17ca944e1bc550baee61078b2` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:31.956090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba7380c17ca944e1bc550baee61078b2)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:31.958467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:32.042319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba7380c17ca944e1bc550baee61078b2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.060957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba7380c17ca944e1bc550baee61078b2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.060957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba7380c17ca944e1bc550baee61078b2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.078725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba7380c17ca944e1bc550baee61078b2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.096309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba7380c17ca944e1bc550baee61078b2'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.102759 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1097/2176] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:31 2022] DELETE /identity/v3/users/ba7380c17ca944e1bc550baee61078b2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:32.107149 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b6390-4807-4cfc-bc79-1a0656fe3b19 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:32.107609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b6390-4807-4cfc-bc79-1a0656fe3b19 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:32.107701 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5b6390-4807-4cfc-bc79-1a0656fe3b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:32.127526 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4c5b6390-4807-4cfc-bc79-1a0656fe3b19 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:32.128944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4c5b6390-4807-4cfc-bc79-1a0656fe3b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:32.197665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1080/2177] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:32.208820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:32.210221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:32.275486 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67639536-6cc8-4142-8483-5688da2316b9 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:32.279388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564dbb8-ad9e-45f5-9d13-aa705e40b071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:32.280027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564dbb8-ad9e-45f5-9d13-aa705e40b071 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:32.280301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564dbb8-ad9e-45f5-9d13-aa705e40b071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:32.280569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2564dbb8-ad9e-45f5-9d13-aa705e40b071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:32.291970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564dbb8-ad9e-45f5-9d13-aa705e40b071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:32.293829 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2564dbb8-ad9e-45f5-9d13-aa705e40b071 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:32.306847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1098/2178] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:32.618435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b6390-4807-4cfc-bc79-1a0656fe3b19 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:32.629910 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4c5b6390-4807-4cfc-bc79-1a0656fe3b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:32.647356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5b6390-4807-4cfc-bc79-1a0656fe3b19 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:32.653174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:32.653808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:32.654082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:32.654334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] PATH_INFO: `/v3/projects/ae11b961d07d4071ba3f2dfc84a0fec7` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:32.655783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae11b961d07d4071ba3f2dfc84a0fec7)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:32.658305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:32.717120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae11b961d07d4071ba3f2dfc84a0fec7', 'request_id': 'req-a097e31b-3620-45a6-bcce-6a0fddc8163f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.725839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae11b961d07d4071ba3f2dfc84a0fec7', 'request_id': 'req-a097e31b-3620-45a6-bcce-6a0fddc8163f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.741665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae11b961d07d4071ba3f2dfc84a0fec7', 'request_id': 'req-a097e31b-3620-45a6-bcce-6a0fddc8163f'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.753692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] The token cache is being invalidate because project ae11b961d07d4071ba3f2dfc84a0fec7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:32.753962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.754602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.756105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1081/2179] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:32 2022] DELETE /identity/v3/projects/ae11b961d07d4071ba3f2dfc84a0fec7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:32.760295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564dbb8-ad9e-45f5-9d13-aa705e40b071 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:32.771201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2564dbb8-ad9e-45f5-9d13-aa705e40b071 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:32.799055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:32.833440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d757327b-0cb2-42d5-9b54-620bd6303430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:32.835096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757327b-0cb2-42d5-9b54-620bd6303430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:32.835096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757327b-0cb2-42d5-9b54-620bd6303430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:32.835488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d757327b-0cb2-42d5-9b54-620bd6303430 admin admin] PATH_INFO: `/v3/users/b93c47ee327544db9b1460c6b4913bad` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:32.836785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757327b-0cb2-42d5-9b54-620bd6303430 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b93c47ee327544db9b1460c6b4913bad)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:32.840790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d757327b-0cb2-42d5-9b54-620bd6303430 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:32.902379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d757327b-0cb2-42d5-9b54-620bd6303430 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b93c47ee327544db9b1460c6b4913bad'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.913061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d757327b-0cb2-42d5-9b54-620bd6303430 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b93c47ee327544db9b1460c6b4913bad'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.914547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d757327b-0cb2-42d5-9b54-620bd6303430 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b93c47ee327544db9b1460c6b4913bad'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.930524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d757327b-0cb2-42d5-9b54-620bd6303430 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b93c47ee327544db9b1460c6b4913bad'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.938387 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d757327b-0cb2-42d5-9b54-620bd6303430 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b93c47ee327544db9b1460c6b4913bad'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:32.945680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1099/2180] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:32 2022] DELETE /identity/v3/users/b93c47ee327544db9b1460c6b4913bad => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:33.380965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:33.382646 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:33.397670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a097e31b-3620-45a6-bcce-6a0fddc8163f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:33.487251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:33.488023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:33.488365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:33.488700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] PATH_INFO: `/v3/projects/0a45a8e107d24325b7f0e0143a0aed5c` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:33.490818 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a45a8e107d24325b7f0e0143a0aed5c)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:33.493364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:33.508679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:33.569859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a45a8e107d24325b7f0e0143a0aed5c', 'request_id': 'req-015b5065-63e2-4509-8d45-3d6b49976818'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:33.582519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a45a8e107d24325b7f0e0143a0aed5c', 'request_id': 'req-015b5065-63e2-4509-8d45-3d6b49976818'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:33.592618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a45a8e107d24325b7f0e0143a0aed5c', 'request_id': 'req-015b5065-63e2-4509-8d45-3d6b49976818'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:33.605175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] The token cache is being invalidate because project 0a45a8e107d24325b7f0e0143a0aed5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:33.605481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:33.606169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-015b5065-63e2-4509-8d45-3d6b49976818 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:33.607787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1082/2181] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:33 2022] DELETE /identity/v3/projects/0a45a8e107d24325b7f0e0143a0aed5c => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:33.613347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5d62d-350d-41fb-8448-8f6a3777c288 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:33.613636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5d62d-350d-41fb-8448-8f6a3777c288 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:33.613906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df5d62d-350d-41fb-8448-8f6a3777c288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:33.633272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-9df5d62d-350d-41fb-8448-8f6a3777c288 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:33.634397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9df5d62d-350d-41fb-8448-8f6a3777c288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:33.698124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1100/2182] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:33.706639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de52e1-0c74-499b-831a-f8ac06fc3f19 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:33.707092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de52e1-0c74-499b-831a-f8ac06fc3f19 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:33.707404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02de52e1-0c74-499b-831a-f8ac06fc3f19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:33.729161 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-02de52e1-0c74-499b-831a-f8ac06fc3f19 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:33.730393 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-02de52e1-0c74-499b-831a-f8ac06fc3f19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:33.740902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1083/2183] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:33.745169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5d62d-350d-41fb-8448-8f6a3777c288 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:33.759094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df5d62d-350d-41fb-8448-8f6a3777c288 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:33.764369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64ed0c-5f09-4818-98c9-9159dafc7a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:33.765020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64ed0c-5f09-4818-98c9-9159dafc7a89 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:33.765299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64ed0c-5f09-4818-98c9-9159dafc7a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:33.766314 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da64ed0c-5f09-4818-98c9-9159dafc7a89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:33.767080 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64ed0c-5f09-4818-98c9-9159dafc7a89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:33.767380 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64ed0c-5f09-4818-98c9-9159dafc7a89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:33.769115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da64ed0c-5f09-4818-98c9-9159dafc7a89 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:33.785699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1101/2184] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:44:33.790273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de52e1-0c74-499b-831a-f8ac06fc3f19 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:33.800691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02de52e1-0c74-499b-831a-f8ac06fc3f19 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:33.805716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc347-693b-46cc-85f9-abc4be2206bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:33.806327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc347-693b-46cc-85f9-abc4be2206bb admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:33.806609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc347-693b-46cc-85f9-abc4be2206bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:33.806881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4fc347-693b-46cc-85f9-abc4be2206bb admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:33.807508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fc347-693b-46cc-85f9-abc4be2206bb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:33.809443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4fc347-693b-46cc-85f9-abc4be2206bb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:33.844884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1084/2185] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:33 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:33.846091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64ed0c-5f09-4818-98c9-9159dafc7a89 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:33.858850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da64ed0c-5f09-4818-98c9-9159dafc7a89 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:33.895730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:33.936088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff356ac-9728-433d-82d0-2472fe53b616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:33.936704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff356ac-9728-433d-82d0-2472fe53b616 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:33.937279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff356ac-9728-433d-82d0-2472fe53b616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:33.937867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff356ac-9728-433d-82d0-2472fe53b616 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:33.938835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff356ac-9728-433d-82d0-2472fe53b616 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:33.945002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff356ac-9728-433d-82d0-2472fe53b616 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:33.987540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1102/2186] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:33 2022] POST /identity/v3/users => generated 293 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:33.989614 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc347-693b-46cc-85f9-abc4be2206bb admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.001262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4fc347-693b-46cc-85f9-abc4be2206bb admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:34.006409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afcfac-a849-4c78-a82f-76366c8dc96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:34.007266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afcfac-a849-4c78-a82f-76366c8dc96e admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.007550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afcfac-a849-4c78-a82f-76366c8dc96e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.007927 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3afcfac-a849-4c78-a82f-76366c8dc96e admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.008571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afcfac-a849-4c78-a82f-76366c8dc96e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:34.010612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3afcfac-a849-4c78-a82f-76366c8dc96e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:34.038368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1085/2187] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:33 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:34.039699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff356ac-9728-433d-82d0-2472fe53b616 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.057840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff356ac-9728-433d-82d0-2472fe53b616 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:34.063790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408d56a5-5cf7-4c23-a419-6aeed08d7cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:34.064401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d56a5-5cf7-4c23-a419-6aeed08d7cdf admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.064903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d56a5-5cf7-4c23-a419-6aeed08d7cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.065325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408d56a5-5cf7-4c23-a419-6aeed08d7cdf admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.066156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d56a5-5cf7-4c23-a419-6aeed08d7cdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:34.067795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408d56a5-5cf7-4c23-a419-6aeed08d7cdf admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:34.078748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1103/2188] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:34.083468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afcfac-a849-4c78-a82f-76366c8dc96e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.095237 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3afcfac-a849-4c78-a82f-76366c8dc96e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:34.101143 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10206e3c-58f7-459c-9120-3577c2542ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:34.102219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10206e3c-58f7-459c-9120-3577c2542ec6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.102697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10206e3c-58f7-459c-9120-3577c2542ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.103215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10206e3c-58f7-459c-9120-3577c2542ec6 admin admin] PATH_INFO: `/v3/projects/d893afa22a584fa0882c1b8b14fa5e9b/users/bbfd01fdbb074190bcc4566631d35a31/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.129550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10206e3c-58f7-459c-9120-3577c2542ec6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d893afa22a584fa0882c1b8b14fa5e9b, user_id=bbfd01fdbb074190bcc4566631d35a31, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:34.133659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10206e3c-58f7-459c-9120-3577c2542ec6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:34.157062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1086/2189] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:34 2022] PUT /identity/v3/projects/d893afa22a584fa0882c1b8b14fa5e9b/users/bbfd01fdbb074190bcc4566631d35a31/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:34.167396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0faa3-1785-4659-a2cf-e38b62414840 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.167878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0faa3-1785-4659-a2cf-e38b62414840 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.168254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce0faa3-1785-4659-a2cf-e38b62414840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.218352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-5ce0faa3-1785-4659-a2cf-e38b62414840 None None] MFA Rules not processed for user `bbfd01fdbb074190bcc4566631d35a31`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:34.219812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5ce0faa3-1785-4659-a2cf-e38b62414840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:34.290796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1104/2190] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:34.295303 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10206e3c-58f7-459c-9120-3577c2542ec6 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.358224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10206e3c-58f7-459c-9120-3577c2542ec6 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:34.363062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcf730-74f8-4f8e-be26-9affecedaed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:34.363678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcf730-74f8-4f8e-be26-9affecedaed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.364028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcf730-74f8-4f8e-be26-9affecedaed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.364218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcf730-74f8-4f8e-be26-9affecedaed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.364888 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcf730-74f8-4f8e-be26-9affecedaed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:34.370460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcf730-74f8-4f8e-be26-9affecedaed9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:34.403371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1087/2191] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:34 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:34.405335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0faa3-1785-4659-a2cf-e38b62414840 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.416616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce0faa3-1785-4659-a2cf-e38b62414840 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:34.479882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52005058-6487-4b36-bf46-49dbe56c485f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:34.480484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52005058-6487-4b36-bf46-49dbe56c485f admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.480782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52005058-6487-4b36-bf46-49dbe56c485f admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.481064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52005058-6487-4b36-bf46-49dbe56c485f admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.481731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52005058-6487-4b36-bf46-49dbe56c485f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:34.483653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52005058-6487-4b36-bf46-49dbe56c485f admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:34.510071 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1105/2192] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:34 2022] POST /identity/v3/users => generated 293 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:34.517894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcf730-74f8-4f8e-be26-9affecedaed9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.529099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcf730-74f8-4f8e-be26-9affecedaed9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:34.534179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728646f5-8ef6-491c-b663-7cf6f309adcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:34.534780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728646f5-8ef6-491c-b663-7cf6f309adcb admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.535057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728646f5-8ef6-491c-b663-7cf6f309adcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.535330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728646f5-8ef6-491c-b663-7cf6f309adcb admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.535973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728646f5-8ef6-491c-b663-7cf6f309adcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:34.537630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728646f5-8ef6-491c-b663-7cf6f309adcb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:34.549695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1088/2193] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:34.553903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52005058-6487-4b36-bf46-49dbe56c485f admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.566790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52005058-6487-4b36-bf46-49dbe56c485f admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:34.571747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e6b65-ec98-4d87-b592-6b44ec031b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:34.572362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e6b65-ec98-4d87-b592-6b44ec031b9c admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.572638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e6b65-ec98-4d87-b592-6b44ec031b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.572944 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768e6b65-ec98-4d87-b592-6b44ec031b9c admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.573589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e6b65-ec98-4d87-b592-6b44ec031b9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:34.575174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768e6b65-ec98-4d87-b592-6b44ec031b9c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:34.586379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1106/2194] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:34.590606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728646f5-8ef6-491c-b663-7cf6f309adcb admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.602185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728646f5-8ef6-491c-b663-7cf6f309adcb admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:34.608313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a555cd-7cbf-4c03-b290-ac92761871aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:34.609102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a555cd-7cbf-4c03-b290-ac92761871aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.610093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a555cd-7cbf-4c03-b290-ac92761871aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.610466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a555cd-7cbf-4c03-b290-ac92761871aa admin admin] PATH_INFO: `/v3/projects/4404ea082ccf4b09988908e7032c3c9d/users/872015e31d924dfca21aa4409bd48a89/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.620878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:34.641794 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a555cd-7cbf-4c03-b290-ac92761871aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=4404ea082ccf4b09988908e7032c3c9d, user_id=872015e31d924dfca21aa4409bd48a89, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:34.642086 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a555cd-7cbf-4c03-b290-ac92761871aa admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:34.660895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1089/2195] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:34 2022] PUT /identity/v3/projects/4404ea082ccf4b09988908e7032c3c9d/users/872015e31d924dfca21aa4409bd48a89/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:34.667808 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e6b65-ec98-4d87-b592-6b44ec031b9c admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.671628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc0435-9020-48fd-a6f1-495ca9115029 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.671959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc0435-9020-48fd-a6f1-495ca9115029 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.672283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bc0435-9020-48fd-a6f1-495ca9115029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.674278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-768e6b65-ec98-4d87-b592-6b44ec031b9c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:34.687275 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768e6b65-ec98-4d87-b592-6b44ec031b9c admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:34.703777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-41bc0435-9020-48fd-a6f1-495ca9115029 None None] MFA Rules not processed for user `872015e31d924dfca21aa4409bd48a89`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:34.705117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-41bc0435-9020-48fd-a6f1-495ca9115029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:34.718475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:34.772778 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9245a3-c44f-4c35-94e6-9934a3f4f52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:34.773425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9245a3-c44f-4c35-94e6-9934a3f4f52b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.773702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9245a3-c44f-4c35-94e6-9934a3f4f52b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.773999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9245a3-c44f-4c35-94e6-9934a3f4f52b admin admin] PATH_INFO: `/v3/users/c972e89c4e154ba3a1c27338e67adf5b` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.775166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9245a3-c44f-4c35-94e6-9934a3f4f52b admin admin] RBAC: Authorizing `identity:delete_user(user_id=c972e89c4e154ba3a1c27338e67adf5b)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:34.777145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9245a3-c44f-4c35-94e6-9934a3f4f52b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:34.787329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1090/2196] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:34 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:34.792645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc0435-9020-48fd-a6f1-495ca9115029 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.804573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bc0435-9020-48fd-a6f1-495ca9115029 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:34.810026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea48d3-f62d-4258-a93d-f74eb90ff46c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:34.810810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea48d3-f62d-4258-a93d-f74eb90ff46c admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.811378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea48d3-f62d-4258-a93d-f74eb90ff46c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.811806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea48d3-f62d-4258-a93d-f74eb90ff46c admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.813087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea48d3-f62d-4258-a93d-f74eb90ff46c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:34.815208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea48d3-f62d-4258-a93d-f74eb90ff46c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:34.842113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6f9245a3-c44f-4c35-94e6-9934a3f4f52b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c972e89c4e154ba3a1c27338e67adf5b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:34.854273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6f9245a3-c44f-4c35-94e6-9934a3f4f52b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c972e89c4e154ba3a1c27338e67adf5b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:34.854273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6f9245a3-c44f-4c35-94e6-9934a3f4f52b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c972e89c4e154ba3a1c27338e67adf5b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:34.855279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1091/2197] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:34 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:34.856222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea48d3-f62d-4258-a93d-f74eb90ff46c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.858113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a6ea48d3-f62d-4258-a93d-f74eb90ff46c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:34.868124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6f9245a3-c44f-4c35-94e6-9934a3f4f52b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c972e89c4e154ba3a1c27338e67adf5b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:34.877025 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6f9245a3-c44f-4c35-94e6-9934a3f4f52b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c972e89c4e154ba3a1c27338e67adf5b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:34.883474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1107/2198] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:34 2022] DELETE /identity/v3/users/c972e89c4e154ba3a1c27338e67adf5b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:34.887933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d477e0-6679-45bd-b6fb-2d15004a2cd5 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:34.888216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d477e0-6679-45bd-b6fb-2d15004a2cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:34.888485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d477e0-6679-45bd-b6fb-2d15004a2cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:34.909423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-48d477e0-6679-45bd-b6fb-2d15004a2cd5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:34.910583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-48d477e0-6679-45bd-b6fb-2d15004a2cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:34.958548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea48d3-f62d-4258-a93d-f74eb90ff46c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:34.978861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1108/2199] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:34.990593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d477e0-6679-45bd-b6fb-2d15004a2cd5 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:34.992024 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-48d477e0-6679-45bd-b6fb-2d15004a2cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:35.030464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09d28-3056-40b0-931f-da00a5234fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:35.031895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09d28-3056-40b0-931f-da00a5234fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:35.031895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09d28-3056-40b0-931f-da00a5234fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:35.032181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d09d28-3056-40b0-931f-da00a5234fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:35.033219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d09d28-3056-40b0-931f-da00a5234fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:35.036124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d09d28-3056-40b0-931f-da00a5234fe9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:35.050108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:35.057781 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:35.076413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1092/2200] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:34 2022] POST /identity/v3/users => generated 300 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:35.077052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d477e0-6679-45bd-b6fb-2d15004a2cd5 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:35.081990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09d28-3056-40b0-931f-da00a5234fe9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:35.082566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c9a3d-48ff-43fb-b191-09198f0e5882 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:35.083285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c9a3d-48ff-43fb-b191-09198f0e5882 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:35.083660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c9a3d-48ff-43fb-b191-09198f0e5882 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:35.084111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91c9a3d-48ff-43fb-b191-09198f0e5882 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:35.095188 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c9a3d-48ff-43fb-b191-09198f0e5882 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:35.101221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91c9a3d-48ff-43fb-b191-09198f0e5882 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:35.101426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d09d28-3056-40b0-931f-da00a5234fe9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:35.108786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1109/2201] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:44:35.116350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3a2dd-80fc-4c4e-bb3f-617ad2312213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:35.116999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3a2dd-80fc-4c4e-bb3f-617ad2312213 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:35.117302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3a2dd-80fc-4c4e-bb3f-617ad2312213 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:35.117573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b3a2dd-80fc-4c4e-bb3f-617ad2312213 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:35.118249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3a2dd-80fc-4c4e-bb3f-617ad2312213 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:35.119843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b3a2dd-80fc-4c4e-bb3f-617ad2312213 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:35.130421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1093/2202] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:35.134722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c9a3d-48ff-43fb-b191-09198f0e5882 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:35.144608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91c9a3d-48ff-43fb-b191-09198f0e5882 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:35.149395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f149e-8ae3-4183-9ef2-e29ec24926f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:35.150139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f149e-8ae3-4183-9ef2-e29ec24926f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:35.150418 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f149e-8ae3-4183-9ef2-e29ec24926f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:35.150691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11f149e-8ae3-4183-9ef2-e29ec24926f1 admin admin] PATH_INFO: `/v3/projects/d90fa75583434a92acb91fb9b6403aa9/users/5638ccbe79db4aa0b543115c9d955113/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:35.165870 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f149e-8ae3-4183-9ef2-e29ec24926f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d90fa75583434a92acb91fb9b6403aa9, user_id=5638ccbe79db4aa0b543115c9d955113, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:35.171772 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c11f149e-8ae3-4183-9ef2-e29ec24926f1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:35.187477 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1110/2203] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:35 2022] PUT /identity/v3/projects/d90fa75583434a92acb91fb9b6403aa9/users/5638ccbe79db4aa0b543115c9d955113/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:44:35.191702 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3a2dd-80fc-4c4e-bb3f-617ad2312213 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:35.202348 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b3a2dd-80fc-4c4e-bb3f-617ad2312213 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:35.293667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeaff737-c268-4d6c-9335-614b0a5e3180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:35.294552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaff737-c268-4d6c-9335-614b0a5e3180 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:35.294666 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaff737-c268-4d6c-9335-614b0a5e3180 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:35.294866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeaff737-c268-4d6c-9335-614b0a5e3180 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:35.295517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaff737-c268-4d6c-9335-614b0a5e3180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:35.297600 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeaff737-c268-4d6c-9335-614b0a5e3180 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:35.308283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1094/2204] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:35 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:35.313206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f149e-8ae3-4183-9ef2-e29ec24926f1 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:35.323443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11f149e-8ae3-4183-9ef2-e29ec24926f1 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:35.329495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9e91d-b2a3-4bd6-9b10-03e076ae92f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:35.330216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9e91d-b2a3-4bd6-9b10-03e076ae92f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:35.330489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9e91d-b2a3-4bd6-9b10-03e076ae92f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:35.330756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b9e91d-b2a3-4bd6-9b10-03e076ae92f3 admin admin] PATH_INFO: `/v3/projects/d90fa75583434a92acb91fb9b6403aa9/users/5638ccbe79db4aa0b543115c9d955113/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:35.332974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9e91d-b2a3-4bd6-9b10-03e076ae92f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d90fa75583434a92acb91fb9b6403aa9, user_id=5638ccbe79db4aa0b543115c9d955113, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:35.335626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b9e91d-b2a3-4bd6-9b10-03e076ae92f3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:35.347590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1111/2205] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:35 2022] PUT /identity/v3/projects/d90fa75583434a92acb91fb9b6403aa9/users/5638ccbe79db4aa0b543115c9d955113/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:35.358410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460857ed-b67f-432c-bb3e-9fef9b2c22cd None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:35.358705 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460857ed-b67f-432c-bb3e-9fef9b2c22cd None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:35.358968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460857ed-b67f-432c-bb3e-9fef9b2c22cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:35.393418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-460857ed-b67f-432c-bb3e-9fef9b2c22cd None None] MFA Rules not processed for user `5638ccbe79db4aa0b543115c9d955113`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:35.394631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-460857ed-b67f-432c-bb3e-9fef9b2c22cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:35.467115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1095/2206] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:35.479574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9e91d-b2a3-4bd6-9b10-03e076ae92f3 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:35.481562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-10b9e91d-b2a3-4bd6-9b10-03e076ae92f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:35.559372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b9e91d-b2a3-4bd6-9b10-03e076ae92f3 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:35.565333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e774eea-7d11-480c-b3cb-59c609b3a9f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:35.565944 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e774eea-7d11-480c-b3cb-59c609b3a9f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:35.566220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e774eea-7d11-480c-b3cb-59c609b3a9f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:35.566488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e774eea-7d11-480c-b3cb-59c609b3a9f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:35.577332 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e774eea-7d11-480c-b3cb-59c609b3a9f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:35.579329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e774eea-7d11-480c-b3cb-59c609b3a9f2 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:35.593052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1112/2207] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:35.853935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460857ed-b67f-432c-bb3e-9fef9b2c22cd None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:35.855477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-460857ed-b67f-432c-bb3e-9fef9b2c22cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:35.951286 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460857ed-b67f-432c-bb3e-9fef9b2c22cd None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:35.957346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:35.958124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:35.958287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:35.958718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] PATH_INFO: `/v3/projects/cd8dd58cf5954b2da64bd9418f52e48b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:35.961355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cd8dd58cf5954b2da64bd9418f52e48b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:35.963737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:35.966706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:36.030118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cd8dd58cf5954b2da64bd9418f52e48b', 'request_id': 'req-23820b07-5b9c-4d4e-a2fd-7e954939f87e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.036740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cd8dd58cf5954b2da64bd9418f52e48b', 'request_id': 'req-23820b07-5b9c-4d4e-a2fd-7e954939f87e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.045815 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cd8dd58cf5954b2da64bd9418f52e48b', 'request_id': 'req-23820b07-5b9c-4d4e-a2fd-7e954939f87e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.060354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] The token cache is being invalidate because project cd8dd58cf5954b2da64bd9418f52e48b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:36.060568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.061233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.063792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1096/2208] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:35 2022] DELETE /identity/v3/projects/cd8dd58cf5954b2da64bd9418f52e48b => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:36.070187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e774eea-7d11-480c-b3cb-59c609b3a9f2 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:36.082833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e774eea-7d11-480c-b3cb-59c609b3a9f2 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:36.151362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:36.152202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:36.152492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:36.152805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] PATH_INFO: `/v3/users/94af4a692f9b4d479a97684d1af72e6e` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:36.154929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94af4a692f9b4d479a97684d1af72e6e)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:36.157024 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:36.219368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94af4a692f9b4d479a97684d1af72e6e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.223474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:36.230340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94af4a692f9b4d479a97684d1af72e6e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.231212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94af4a692f9b4d479a97684d1af72e6e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.253602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94af4a692f9b4d479a97684d1af72e6e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.262151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94af4a692f9b4d479a97684d1af72e6e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.268108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:36.272650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1113/2209] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:36 2022] DELETE /identity/v3/users/94af4a692f9b4d479a97684d1af72e6e => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:36.272915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:36.336656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23820b07-5b9c-4d4e-a2fd-7e954939f87e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:36.400179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575cec9-cf3b-4b55-80b8-8256800661ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:36.400844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575cec9-cf3b-4b55-80b8-8256800661ba service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:36.401136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575cec9-cf3b-4b55-80b8-8256800661ba service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:36.401408 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2575cec9-cf3b-4b55-80b8-8256800661ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:36.403219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2575cec9-cf3b-4b55-80b8-8256800661ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:36.475978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575cec9-cf3b-4b55-80b8-8256800661ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:36.477531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2575cec9-cf3b-4b55-80b8-8256800661ba service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:36.487151 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1097/2210] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:36 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:44:36.588130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:36.590170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:36.720522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879973ef-ab0d-4e3b-bb37-12caa0011390 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:36.726080 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeed514-936e-461d-b67d-652481225078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:36.726952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeed514-936e-461d-b67d-652481225078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:36.727227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeed514-936e-461d-b67d-652481225078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:36.727491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeed514-936e-461d-b67d-652481225078 admin admin] PATH_INFO: `/v3/projects/0c79723fd89b4ed0b2236b17f978ee25` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:36.728874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeed514-936e-461d-b67d-652481225078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c79723fd89b4ed0b2236b17f978ee25)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:36.731521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeed514-936e-461d-b67d-652481225078 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:36.814435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-deeed514-936e-461d-b67d-652481225078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c79723fd89b4ed0b2236b17f978ee25', 'request_id': 'req-deeed514-936e-461d-b67d-652481225078'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.826194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-deeed514-936e-461d-b67d-652481225078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c79723fd89b4ed0b2236b17f978ee25', 'request_id': 'req-deeed514-936e-461d-b67d-652481225078'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.847022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-deeed514-936e-461d-b67d-652481225078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c79723fd89b4ed0b2236b17f978ee25', 'request_id': 'req-deeed514-936e-461d-b67d-652481225078'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.860513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-deeed514-936e-461d-b67d-652481225078 admin admin] The token cache is being invalidate because project 0c79723fd89b4ed0b2236b17f978ee25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:36.861176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-deeed514-936e-461d-b67d-652481225078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.865541 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-deeed514-936e-461d-b67d-652481225078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:36.867229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1114/2211] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:36 2022] DELETE /identity/v3/projects/0c79723fd89b4ed0b2236b17f978ee25 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:36.876052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575cec9-cf3b-4b55-80b8-8256800661ba service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:36.882314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:36.888182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2575cec9-cf3b-4b55-80b8-8256800661ba service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:36.997078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f955ae2e-f28f-48f8-951b-30b434168d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:36.997757 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ae2e-f28f-48f8-951b-30b434168d75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:36.998111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ae2e-f28f-48f8-951b-30b434168d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:36.998454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f955ae2e-f28f-48f8-951b-30b434168d75 admin admin] PATH_INFO: `/v3/users/d64e87e0df9e41b8b5d62d0758ec3842` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:36.999718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955ae2e-f28f-48f8-951b-30b434168d75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d64e87e0df9e41b8b5d62d0758ec3842)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:37.001978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f955ae2e-f28f-48f8-951b-30b434168d75 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:37.052106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f955ae2e-f28f-48f8-951b-30b434168d75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd64e87e0df9e41b8b5d62d0758ec3842'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:37.052866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f955ae2e-f28f-48f8-951b-30b434168d75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd64e87e0df9e41b8b5d62d0758ec3842'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:37.059987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f955ae2e-f28f-48f8-951b-30b434168d75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd64e87e0df9e41b8b5d62d0758ec3842'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:37.067566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f955ae2e-f28f-48f8-951b-30b434168d75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd64e87e0df9e41b8b5d62d0758ec3842'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:37.079828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f955ae2e-f28f-48f8-951b-30b434168d75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd64e87e0df9e41b8b5d62d0758ec3842'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:37.085635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1098/2212] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:36 2022] DELETE /identity/v3/users/d64e87e0df9e41b8b5d62d0758ec3842 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:44:37.431072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeed514-936e-461d-b67d-652481225078 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:37.432520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-deeed514-936e-461d-b67d-652481225078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:37.457184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeed514-936e-461d-b67d-652481225078 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:37.517551 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:37.518176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:37.518450 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:37.518713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] PATH_INFO: `/v3/projects/f0448bb0d06047d5a4175eb84791f9ba` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:37.520127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0448bb0d06047d5a4175eb84791f9ba)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:37.522122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:37.523607 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:37.577241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0448bb0d06047d5a4175eb84791f9ba', 'request_id': 'req-016973c2-6745-4427-bb23-bfe9887ee442'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:37.585526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0448bb0d06047d5a4175eb84791f9ba', 'request_id': 'req-016973c2-6745-4427-bb23-bfe9887ee442'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:37.591353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0448bb0d06047d5a4175eb84791f9ba', 'request_id': 'req-016973c2-6745-4427-bb23-bfe9887ee442'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:37.602438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] The token cache is being invalidate because project f0448bb0d06047d5a4175eb84791f9ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:37.602732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:37.603350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-016973c2-6745-4427-bb23-bfe9887ee442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:37.604866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1115/2213] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:37 2022] DELETE /identity/v3/projects/f0448bb0d06047d5a4175eb84791f9ba => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:37.611437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e61852-f76f-409f-ae50-c57b1db56d16 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:37.611902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e61852-f76f-409f-ae50-c57b1db56d16 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:37.612007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e61852-f76f-409f-ae50-c57b1db56d16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:37.632162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-35e61852-f76f-409f-ae50-c57b1db56d16 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:37.633385 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-35e61852-f76f-409f-ae50-c57b1db56d16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:37.694811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1099/2214] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:37.703623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1518d4-fc0d-4c58-a5b3-246e05dc8db7 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:37.703907 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1518d4-fc0d-4c58-a5b3-246e05dc8db7 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:37.704204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1518d4-fc0d-4c58-a5b3-246e05dc8db7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:37.723988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7e1518d4-fc0d-4c58-a5b3-246e05dc8db7 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:37.725240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7e1518d4-fc0d-4c58-a5b3-246e05dc8db7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:37.737930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1116/2215] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:37.738603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e61852-f76f-409f-ae50-c57b1db56d16 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:37.748785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e61852-f76f-409f-ae50-c57b1db56d16 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:37.755408 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fef557-5073-4567-b215-b4aeb5d8d40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:37.756017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fef557-5073-4567-b215-b4aeb5d8d40b admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:37.756289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fef557-5073-4567-b215-b4aeb5d8d40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:37.756554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fef557-5073-4567-b215-b4aeb5d8d40b admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:37.757669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fef557-5073-4567-b215-b4aeb5d8d40b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:37.757669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fef557-5073-4567-b215-b4aeb5d8d40b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:37.759081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fef557-5073-4567-b215-b4aeb5d8d40b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:37.771775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1100/2216] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:44:37.776231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1518d4-fc0d-4c58-a5b3-246e05dc8db7 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:37.785885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1518d4-fc0d-4c58-a5b3-246e05dc8db7 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:37.790520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c8625-762f-40ae-b66e-b7688d2bc7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:37.791124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c8625-762f-40ae-b66e-b7688d2bc7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:37.791395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c8625-762f-40ae-b66e-b7688d2bc7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:37.791664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026c8625-762f-40ae-b66e-b7688d2bc7e3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:37.792310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c8625-762f-40ae-b66e-b7688d2bc7e3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:37.794279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026c8625-762f-40ae-b66e-b7688d2bc7e3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:37.827849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1117/2217] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:37 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:37.829307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fef557-5073-4567-b215-b4aeb5d8d40b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:37.840011 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fef557-5073-4567-b215-b4aeb5d8d40b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:37.845165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:37.909899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba06db-5128-4c80-a125-e51a1dcb2262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:37.910648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba06db-5128-4c80-a125-e51a1dcb2262 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:37.910931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba06db-5128-4c80-a125-e51a1dcb2262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:37.911202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ba06db-5128-4c80-a125-e51a1dcb2262 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:37.911842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba06db-5128-4c80-a125-e51a1dcb2262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:37.913807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ba06db-5128-4c80-a125-e51a1dcb2262 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:37.943216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1101/2218] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:37 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:37.944830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c8625-762f-40ae-b66e-b7688d2bc7e3 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:37.962113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026c8625-762f-40ae-b66e-b7688d2bc7e3 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:37.967777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e9c66-9216-480b-a074-c1e38b9ebd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:37.968385 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e9c66-9216-480b-a074-c1e38b9ebd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:37.969242 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e9c66-9216-480b-a074-c1e38b9ebd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:37.969515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49e9c66-9216-480b-a074-c1e38b9ebd04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:37.970260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e9c66-9216-480b-a074-c1e38b9ebd04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:37.971843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49e9c66-9216-480b-a074-c1e38b9ebd04 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:37.984892 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1118/2219] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:37.990323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba06db-5128-4c80-a125-e51a1dcb2262 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:38.000883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ba06db-5128-4c80-a125-e51a1dcb2262 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:38.005698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced2098a-3164-4611-992b-9a5a96c0da19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:38.006298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2098a-3164-4611-992b-9a5a96c0da19 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:38.006572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2098a-3164-4611-992b-9a5a96c0da19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:38.006956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced2098a-3164-4611-992b-9a5a96c0da19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:38.007707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2098a-3164-4611-992b-9a5a96c0da19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:38.009313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced2098a-3164-4611-992b-9a5a96c0da19 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:38.020358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1102/2220] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:38.024939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e9c66-9216-480b-a074-c1e38b9ebd04 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:38.036814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49e9c66-9216-480b-a074-c1e38b9ebd04 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:38.041600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805fc60-2170-4890-9119-1b5f8bcee443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:38.042300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805fc60-2170-4890-9119-1b5f8bcee443 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:38.042578 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805fc60-2170-4890-9119-1b5f8bcee443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:38.042850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c805fc60-2170-4890-9119-1b5f8bcee443 admin admin] PATH_INFO: `/v3/projects/ad0aee19677643cbbaeff056c8e5a2c4/users/1e73bc19084a4bab9727b9dda49b37c6/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:38.061007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805fc60-2170-4890-9119-1b5f8bcee443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad0aee19677643cbbaeff056c8e5a2c4, user_id=1e73bc19084a4bab9727b9dda49b37c6, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:38.064328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c805fc60-2170-4890-9119-1b5f8bcee443 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:38.082187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1119/2221] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:38 2022] PUT /identity/v3/projects/ad0aee19677643cbbaeff056c8e5a2c4/users/1e73bc19084a4bab9727b9dda49b37c6/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:38.092380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c6c35-7e51-4ebd-a4c8-23687fe5d5f0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:38.092661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c6c35-7e51-4ebd-a4c8-23687fe5d5f0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:38.092961 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4c6c35-7e51-4ebd-a4c8-23687fe5d5f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:38.132093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5e4c6c35-7e51-4ebd-a4c8-23687fe5d5f0 None None] MFA Rules not processed for user `1e73bc19084a4bab9727b9dda49b37c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:38.133377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5e4c6c35-7e51-4ebd-a4c8-23687fe5d5f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:38.211689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1103/2222] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:38.223342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805fc60-2170-4890-9119-1b5f8bcee443 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:38.225108 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c805fc60-2170-4890-9119-1b5f8bcee443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:38.284715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c805fc60-2170-4890-9119-1b5f8bcee443 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:38.288623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75f400-0825-4018-9d2b-109ea8e4bc53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:38.289246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f400-0825-4018-9d2b-109ea8e4bc53 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:38.289528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f400-0825-4018-9d2b-109ea8e4bc53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:38.289799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc75f400-0825-4018-9d2b-109ea8e4bc53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:38.298757 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75f400-0825-4018-9d2b-109ea8e4bc53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:38.300179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc75f400-0825-4018-9d2b-109ea8e4bc53 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:38.310776 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1120/2223] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:39.688300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28d15-c105-46f9-974d-73fdf6f80825 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:39.688836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28d15-c105-46f9-974d-73fdf6f80825 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:39.688977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c28d15-c105-46f9-974d-73fdf6f80825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:39.707752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-52c28d15-c105-46f9-974d-73fdf6f80825 None None] MFA Rules not processed for user `1e73bc19084a4bab9727b9dda49b37c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:39.709120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-52c28d15-c105-46f9-974d-73fdf6f80825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:39.715979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1104/2224] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:40.856917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75f400-0825-4018-9d2b-109ea8e4bc53 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:40.867291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc75f400-0825-4018-9d2b-109ea8e4bc53 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:40.871435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9531aaa-a6fb-41f4-93fc-8dc7405a62d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:40.872151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9531aaa-a6fb-41f4-93fc-8dc7405a62d5 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:40.872502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9531aaa-a6fb-41f4-93fc-8dc7405a62d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:40.872906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9531aaa-a6fb-41f4-93fc-8dc7405a62d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:40.874724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c9531aaa-a6fb-41f4-93fc-8dc7405a62d5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:40.907453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:40.949478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9531aaa-a6fb-41f4-93fc-8dc7405a62d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:40.950964 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9531aaa-a6fb-41f4-93fc-8dc7405a62d5 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:40.963838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1121/2225] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:43.349624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c28d15-c105-46f9-974d-73fdf6f80825 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:43.351174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-52c28d15-c105-46f9-974d-73fdf6f80825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:43.461923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c28d15-c105-46f9-974d-73fdf6f80825 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:43.476034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:43.477049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:43.478466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:43.479110 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] PATH_INFO: `/v3/users/bbfd01fdbb074190bcc4566631d35a31` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:43.482551 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbfd01fdbb074190bcc4566631d35a31)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:43.490194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:43.534969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:43.613861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbfd01fdbb074190bcc4566631d35a31'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:43.614626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbfd01fdbb074190bcc4566631d35a31'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:43.631004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbfd01fdbb074190bcc4566631d35a31'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:43.654923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbfd01fdbb074190bcc4566631d35a31'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:43.689420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbfd01fdbb074190bcc4566631d35a31'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:43.710591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1105/2226] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:43 2022] DELETE /identity/v3/users/bbfd01fdbb074190bcc4566631d35a31 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:43.716280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b953bd-b6df-4869-bff2-6d28c0634c60 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:43.716716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b953bd-b6df-4869-bff2-6d28c0634c60 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:43.716981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b953bd-b6df-4869-bff2-6d28c0634c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:43.742579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-62b953bd-b6df-4869-bff2-6d28c0634c60 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:43.744034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-62b953bd-b6df-4869-bff2-6d28c0634c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:43.810848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1122/2227] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:43.821060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:43.822672 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:43.890214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75fa5f0-4d87-4379-a07e-6346fdce2f9a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:43.901520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7525d-6f3a-4fca-8864-d56f4b94f80a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:43.902536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7525d-6f3a-4fca-8864-d56f4b94f80a service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:43.902773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7525d-6f3a-4fca-8864-d56f4b94f80a service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:43.903038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc7525d-6f3a-4fca-8864-d56f4b94f80a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:43.915429 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7525d-6f3a-4fca-8864-d56f4b94f80a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:43.916686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc7525d-6f3a-4fca-8864-d56f4b94f80a service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:43.928185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1106/2228] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:44.257649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b953bd-b6df-4869-bff2-6d28c0634c60 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:44.259452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-62b953bd-b6df-4869-bff2-6d28c0634c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:44.296281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b953bd-b6df-4869-bff2-6d28c0634c60 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:44.320323 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:44.321613 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:44.322025 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:44.322615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] PATH_INFO: `/v3/projects/d893afa22a584fa0882c1b8b14fa5e9b` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:44.324082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] RBAC: Authorizing `identity:delete_project(project_id=d893afa22a584fa0882c1b8b14fa5e9b)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:44.329364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:44.390962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd893afa22a584fa0882c1b8b14fa5e9b', 'request_id': 'req-268f1e3a-117a-40ec-91f4-a6dca0fb245c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:44.405240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd893afa22a584fa0882c1b8b14fa5e9b', 'request_id': 'req-268f1e3a-117a-40ec-91f4-a6dca0fb245c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:44.413401 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd893afa22a584fa0882c1b8b14fa5e9b', 'request_id': 'req-268f1e3a-117a-40ec-91f4-a6dca0fb245c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:44.427761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] The token cache is being invalidate because project d893afa22a584fa0882c1b8b14fa5e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:44.428205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:44.429511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:44.431467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1123/2229] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:44 2022] DELETE /identity/v3/projects/d893afa22a584fa0882c1b8b14fa5e9b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:44.435753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7525d-6f3a-4fca-8864-d56f4b94f80a service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:44.448498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc7525d-6f3a-4fca-8864-d56f4b94f80a service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:44.514278 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:44.515274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:44.515785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:44.516123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] PATH_INFO: `/v3/users/872015e31d924dfca21aa4409bd48a89` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:44.517463 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=872015e31d924dfca21aa4409bd48a89)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:44.519529 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:44.571443 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:44.590516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '872015e31d924dfca21aa4409bd48a89'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:44.591321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '872015e31d924dfca21aa4409bd48a89'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:44.603107 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '872015e31d924dfca21aa4409bd48a89'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:44.612159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '872015e31d924dfca21aa4409bd48a89'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:44.659368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '872015e31d924dfca21aa4409bd48a89'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:44.667391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1107/2230] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:44 2022] DELETE /identity/v3/users/872015e31d924dfca21aa4409bd48a89 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:45.064553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:45.066702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:45.083485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268f1e3a-117a-40ec-91f4-a6dca0fb245c admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:45.205385 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:45.219556 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:45.220195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:45.220468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:45.220734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] PATH_INFO: `/v3/projects/4404ea082ccf4b09988908e7032c3c9d` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:45.221913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4404ea082ccf4b09988908e7032c3c9d)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:45.223827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:45.273099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4404ea082ccf4b09988908e7032c3c9d', 'request_id': 'req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.282444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4404ea082ccf4b09988908e7032c3c9d', 'request_id': 'req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.290512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4404ea082ccf4b09988908e7032c3c9d', 'request_id': 'req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.302184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] The token cache is being invalidate because project 4404ea082ccf4b09988908e7032c3c9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:45.302465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.303140 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.304608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1124/2231] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:45 2022] DELETE /identity/v3/projects/4404ea082ccf4b09988908e7032c3c9d => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:45.308643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:45.310117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:45.323743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d29c01-0219-4d91-8efc-cfe6341cca74 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:45.382391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcaa56a-2a2f-4e8e-bec4-8e4d7fd07525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:45.383017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcaa56a-2a2f-4e8e-bec4-8e4d7fd07525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:45.383295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcaa56a-2a2f-4e8e-bec4-8e4d7fd07525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:45.383567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcaa56a-2a2f-4e8e-bec4-8e4d7fd07525 admin admin] PATH_INFO: `/v3/users/5638ccbe79db4aa0b543115c9d955113` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:45.384687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcaa56a-2a2f-4e8e-bec4-8e4d7fd07525 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5638ccbe79db4aa0b543115c9d955113)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:45.386639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcaa56a-2a2f-4e8e-bec4-8e4d7fd07525 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:45.432872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-cbcaa56a-2a2f-4e8e-bec4-8e4d7fd07525 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5638ccbe79db4aa0b543115c9d955113'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.433691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-cbcaa56a-2a2f-4e8e-bec4-8e4d7fd07525 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5638ccbe79db4aa0b543115c9d955113'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.443946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-cbcaa56a-2a2f-4e8e-bec4-8e4d7fd07525 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5638ccbe79db4aa0b543115c9d955113'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.451716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-cbcaa56a-2a2f-4e8e-bec4-8e4d7fd07525 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5638ccbe79db4aa0b543115c9d955113'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.464004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-cbcaa56a-2a2f-4e8e-bec4-8e4d7fd07525 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5638ccbe79db4aa0b543115c9d955113'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.470528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1108/2232] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:45 2022] DELETE /identity/v3/users/5638ccbe79db4aa0b543115c9d955113 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:45.770254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:45.771687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:45.791709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c047cc-c358-4dd7-b7f7-18dd6bdf9c6e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:45.861927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:45.862548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:45.862822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:45.863090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] PATH_INFO: `/v3/projects/d90fa75583434a92acb91fb9b6403aa9` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:45.864215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d90fa75583434a92acb91fb9b6403aa9)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:45.866127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:45.911260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd90fa75583434a92acb91fb9b6403aa9', 'request_id': 'req-31c98f09-d170-4472-9c2f-87ecc1885095'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.919614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd90fa75583434a92acb91fb9b6403aa9', 'request_id': 'req-31c98f09-d170-4472-9c2f-87ecc1885095'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.927028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd90fa75583434a92acb91fb9b6403aa9', 'request_id': 'req-31c98f09-d170-4472-9c2f-87ecc1885095'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.940546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] The token cache is being invalidate because project d90fa75583434a92acb91fb9b6403aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:45.940891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.941621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-31c98f09-d170-4472-9c2f-87ecc1885095 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:45.943191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1125/2233] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:45 2022] DELETE /identity/v3/projects/d90fa75583434a92acb91fb9b6403aa9 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:45.949096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e5d3b-5d7f-43fa-8897-011b32177e80 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:45.949386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e5d3b-5d7f-43fa-8897-011b32177e80 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:45.949661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e5d3b-5d7f-43fa-8897-011b32177e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:45.970596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-731e5d3b-5d7f-43fa-8897-011b32177e80 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:45.971872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-731e5d3b-5d7f-43fa-8897-011b32177e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:46.003642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:46.067669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1109/2234] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:46.077961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290ffce-d5f6-403e-ae81-911a4a2bee10 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:46.079396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290ffce-d5f6-403e-ae81-911a4a2bee10 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:46.079872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290ffce-d5f6-403e-ae81-911a4a2bee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:46.105616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-3290ffce-d5f6-403e-ae81-911a4a2bee10 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:46.106889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3290ffce-d5f6-403e-ae81-911a4a2bee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:46.118393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1126/2235] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:46.122802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e5d3b-5d7f-43fa-8897-011b32177e80 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:46.134330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e5d3b-5d7f-43fa-8897-011b32177e80 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:46.141007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e3095-9ab8-4116-933b-9b02be2552f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:46.141626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e3095-9ab8-4116-933b-9b02be2552f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:46.141900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e3095-9ab8-4116-933b-9b02be2552f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:46.142173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5e3095-9ab8-4116-933b-9b02be2552f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:46.142830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e3095-9ab8-4116-933b-9b02be2552f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:46.143134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e3095-9ab8-4116-933b-9b02be2552f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:46.144730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5e3095-9ab8-4116-933b-9b02be2552f8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:46.158656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1110/2236] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:44:46.162965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290ffce-d5f6-403e-ae81-911a4a2bee10 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:46.172940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:46.180151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290ffce-d5f6-403e-ae81-911a4a2bee10 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:46.190782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b5497-bf4e-486e-9377-89a673062c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:46.193019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b5497-bf4e-486e-9377-89a673062c75 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:46.195015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b5497-bf4e-486e-9377-89a673062c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:46.195591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6b5497-bf4e-486e-9377-89a673062c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:46.196923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b5497-bf4e-486e-9377-89a673062c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:46.200466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6b5497-bf4e-486e-9377-89a673062c75 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:46.241550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1127/2237] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:46 2022] POST /identity/v3/projects => generated 377 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:46.245817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e3095-9ab8-4116-933b-9b02be2552f8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:46.261847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5e3095-9ab8-4116-933b-9b02be2552f8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:46.380741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2a95f-ad56-4ea6-a7e7-34f24dd83b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:46.382131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2a95f-ad56-4ea6-a7e7-34f24dd83b13 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:46.382669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2a95f-ad56-4ea6-a7e7-34f24dd83b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:46.383092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d2a95f-ad56-4ea6-a7e7-34f24dd83b13 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:46.384354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2a95f-ad56-4ea6-a7e7-34f24dd83b13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:46.387262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d2a95f-ad56-4ea6-a7e7-34f24dd83b13 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:46.440426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1111/2238] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:46 2022] POST /identity/v3/users => generated 289 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:46.442356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b5497-bf4e-486e-9377-89a673062c75 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:46.462322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6b5497-bf4e-486e-9377-89a673062c75 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:46.467154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98d649-e127-40ae-8c7b-39aea84fa7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:46.467758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d649-e127-40ae-8c7b-39aea84fa7f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:46.468229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d649-e127-40ae-8c7b-39aea84fa7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:46.468933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a98d649-e127-40ae-8c7b-39aea84fa7f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:46.469744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98d649-e127-40ae-8c7b-39aea84fa7f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:46.471505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a98d649-e127-40ae-8c7b-39aea84fa7f7 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:46.483527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1128/2239] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:46.488094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2a95f-ad56-4ea6-a7e7-34f24dd83b13 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:46.505972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d2a95f-ad56-4ea6-a7e7-34f24dd83b13 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:46.512837 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499b517-92bc-453e-ac3a-ce856c5c3f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:46.513666 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499b517-92bc-453e-ac3a-ce856c5c3f85 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:46.514366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499b517-92bc-453e-ac3a-ce856c5c3f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:46.514814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499b517-92bc-453e-ac3a-ce856c5c3f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:46.516396 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499b517-92bc-453e-ac3a-ce856c5c3f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:46.518330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499b517-92bc-453e-ac3a-ce856c5c3f85 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:46.535106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1112/2240] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:46.539853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98d649-e127-40ae-8c7b-39aea84fa7f7 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:46.557503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a98d649-e127-40ae-8c7b-39aea84fa7f7 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:46.563662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a50b1-3a54-4267-b2aa-4f3f43670b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:46.564341 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a50b1-3a54-4267-b2aa-4f3f43670b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:46.564621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a50b1-3a54-4267-b2aa-4f3f43670b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:46.564929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194a50b1-3a54-4267-b2aa-4f3f43670b0a admin admin] PATH_INFO: `/v3/projects/8eff3878504c40b08a176d29e30764bf/users/b5643881ae9448a4be3575c202e310f6/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:46.582653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a50b1-3a54-4267-b2aa-4f3f43670b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8eff3878504c40b08a176d29e30764bf, user_id=b5643881ae9448a4be3575c202e310f6, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:46.592200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194a50b1-3a54-4267-b2aa-4f3f43670b0a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:46.614362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1129/2241] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:46 2022] PUT /identity/v3/projects/8eff3878504c40b08a176d29e30764bf/users/b5643881ae9448a4be3575c202e310f6/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:46.626784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eab4c2-95bc-4f3b-95e5-593d9e69bd71 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:46.627691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eab4c2-95bc-4f3b-95e5-593d9e69bd71 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:46.627691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eab4c2-95bc-4f3b-95e5-593d9e69bd71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:46.656167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c7eab4c2-95bc-4f3b-95e5-593d9e69bd71 None None] MFA Rules not processed for user `b5643881ae9448a4be3575c202e310f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:46.657666 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c7eab4c2-95bc-4f3b-95e5-593d9e69bd71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:46.748608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1113/2242] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:46 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:46.753800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a50b1-3a54-4267-b2aa-4f3f43670b0a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:46.826136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194a50b1-3a54-4267-b2aa-4f3f43670b0a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:46.847250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b01d36-eedd-481c-939a-af7b777428ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:46.847933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b01d36-eedd-481c-939a-af7b777428ca admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:46.848315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b01d36-eedd-481c-939a-af7b777428ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:46.852310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b01d36-eedd-481c-939a-af7b777428ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:46.853102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b01d36-eedd-481c-939a-af7b777428ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:46.855111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b01d36-eedd-481c-939a-af7b777428ca admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:46.885685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1130/2243] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:46 2022] POST /identity/v3/projects => generated 377 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:46.889635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eab4c2-95bc-4f3b-95e5-593d9e69bd71 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:46.901531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eab4c2-95bc-4f3b-95e5-593d9e69bd71 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:46.935327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:46.969837 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df893841-dea5-404d-8cc5-11f102053d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:46.970359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df893841-dea5-404d-8cc5-11f102053d6b admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:46.970631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df893841-dea5-404d-8cc5-11f102053d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:46.970897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df893841-dea5-404d-8cc5-11f102053d6b admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:46.971513 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df893841-dea5-404d-8cc5-11f102053d6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:46.973740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df893841-dea5-404d-8cc5-11f102053d6b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:46.998959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1114/2244] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:46 2022] POST /identity/v3/users => generated 289 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:47.003112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b01d36-eedd-481c-939a-af7b777428ca admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.013236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b01d36-eedd-481c-939a-af7b777428ca admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.017786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a1c07-2aa0-4151-89d1-6a719bd41876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.018367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a1c07-2aa0-4151-89d1-6a719bd41876 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.018642 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a1c07-2aa0-4151-89d1-6a719bd41876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.018907 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46a1c07-2aa0-4151-89d1-6a719bd41876 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.019529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46a1c07-2aa0-4151-89d1-6a719bd41876 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:47.021424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46a1c07-2aa0-4151-89d1-6a719bd41876 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:47.033145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1131/2245] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:47.037471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df893841-dea5-404d-8cc5-11f102053d6b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.048571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df893841-dea5-404d-8cc5-11f102053d6b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.053256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b9ac7-86bc-46d4-905f-0025a503150c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.053847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b9ac7-86bc-46d4-905f-0025a503150c admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.054124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b9ac7-86bc-46d4-905f-0025a503150c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.054396 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76b9ac7-86bc-46d4-905f-0025a503150c admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.055030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b9ac7-86bc-46d4-905f-0025a503150c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:47.056617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76b9ac7-86bc-46d4-905f-0025a503150c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:47.066778 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1115/2246] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:47.070917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a1c07-2aa0-4151-89d1-6a719bd41876 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.080788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46a1c07-2aa0-4151-89d1-6a719bd41876 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.085363 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f45cfad-0229-402b-9ad6-9b92a02ca583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.085998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45cfad-0229-402b-9ad6-9b92a02ca583 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.086270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45cfad-0229-402b-9ad6-9b92a02ca583 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.086722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f45cfad-0229-402b-9ad6-9b92a02ca583 admin admin] PATH_INFO: `/v3/projects/7a1b2a6c1a4e4edaa1a40a3d424e75ba/users/42a117803c3142fd860c9fba32aca95b/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.101945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45cfad-0229-402b-9ad6-9b92a02ca583 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a1b2a6c1a4e4edaa1a40a3d424e75ba, user_id=42a117803c3142fd860c9fba32aca95b, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:47.104645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f45cfad-0229-402b-9ad6-9b92a02ca583 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:47.118992 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1132/2247] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:47 2022] PUT /identity/v3/projects/7a1b2a6c1a4e4edaa1a40a3d424e75ba/users/42a117803c3142fd860c9fba32aca95b/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:47.120464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b9ac7-86bc-46d4-905f-0025a503150c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.122184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f76b9ac7-86bc-46d4-905f-0025a503150c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:47.129778 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ff865-66d5-479f-beb6-4052e7eba4df None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.130065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ff865-66d5-479f-beb6-4052e7eba4df None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.130338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4ff865-66d5-479f-beb6-4052e7eba4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.135611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76b9ac7-86bc-46d4-905f-0025a503150c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.158622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-5a4ff865-66d5-479f-beb6-4052e7eba4df None None] MFA Rules not processed for user `42a117803c3142fd860c9fba32aca95b`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:47.159835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5a4ff865-66d5-479f-beb6-4052e7eba4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:47.198924 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b21c0f9-60d8-4978-ab2b-57c40721075e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.199605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21c0f9-60d8-4978-ab2b-57c40721075e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.199877 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21c0f9-60d8-4978-ab2b-57c40721075e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.200252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b21c0f9-60d8-4978-ab2b-57c40721075e admin admin] PATH_INFO: `/v3/users/3fe8002c0b3545f291f4b0634e6f0302` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.201603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21c0f9-60d8-4978-ab2b-57c40721075e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fe8002c0b3545f291f4b0634e6f0302)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:47.203851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b21c0f9-60d8-4978-ab2b-57c40721075e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:47.223100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:47.264463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1133/2248] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:47.268586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ff865-66d5-479f-beb6-4052e7eba4df None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.269354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1b21c0f9-60d8-4978-ab2b-57c40721075e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fe8002c0b3545f291f4b0634e6f0302'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:47.270024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1b21c0f9-60d8-4978-ab2b-57c40721075e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fe8002c0b3545f291f4b0634e6f0302'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:47.279903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4ff865-66d5-479f-beb6-4052e7eba4df None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.282949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1b21c0f9-60d8-4978-ab2b-57c40721075e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fe8002c0b3545f291f4b0634e6f0302'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:47.286005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ec206-f0bc-4cfe-bb04-67e2a7fb52f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.286620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ec206-f0bc-4cfe-bb04-67e2a7fb52f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.286881 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ec206-f0bc-4cfe-bb04-67e2a7fb52f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.287150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3ec206-f0bc-4cfe-bb04-67e2a7fb52f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.287777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ec206-f0bc-4cfe-bb04-67e2a7fb52f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:47.289871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3ec206-f0bc-4cfe-bb04-67e2a7fb52f3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:47.290487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1b21c0f9-60d8-4978-ab2b-57c40721075e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fe8002c0b3545f291f4b0634e6f0302'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:47.303301 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-1b21c0f9-60d8-4978-ab2b-57c40721075e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fe8002c0b3545f291f4b0634e6f0302'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:47.309349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1116/2249] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:47 2022] DELETE /identity/v3/users/3fe8002c0b3545f291f4b0634e6f0302 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:47.314141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815ad91-2e7a-4589-ba54-41518a3cca2c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.314567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815ad91-2e7a-4589-ba54-41518a3cca2c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.314847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d815ad91-2e7a-4589-ba54-41518a3cca2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.322670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1134/2250] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:47 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:47.323497 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ec206-f0bc-4cfe-bb04-67e2a7fb52f3 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.325105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5d3ec206-f0bc-4cfe-bb04-67e2a7fb52f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:47.334420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d815ad91-2e7a-4589-ba54-41518a3cca2c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:47.335583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d815ad91-2e7a-4589-ba54-41518a3cca2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:47.353189 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:47.393450 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3ec206-f0bc-4cfe-bb04-67e2a7fb52f3 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.401716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad975ab-3d1d-4195-beb4-38535d28653a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.402429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad975ab-3d1d-4195-beb4-38535d28653a admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.402783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad975ab-3d1d-4195-beb4-38535d28653a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.403084 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad975ab-3d1d-4195-beb4-38535d28653a admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.403808 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad975ab-3d1d-4195-beb4-38535d28653a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:47.406014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad975ab-3d1d-4195-beb4-38535d28653a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:47.413868 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1117/2251] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:44:47.421782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815ad91-2e7a-4589-ba54-41518a3cca2c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.423424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d815ad91-2e7a-4589-ba54-41518a3cca2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:47.430329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1135/2252] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:47 2022] POST /identity/v3/users => generated 295 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:47.434363 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad975ab-3d1d-4195-beb4-38535d28653a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.451277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad975ab-3d1d-4195-beb4-38535d28653a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.458170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccc842-d555-404f-b7c5-9b85971457f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.458915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccc842-d555-404f-b7c5-9b85971457f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.459265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccc842-d555-404f-b7c5-9b85971457f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.459585 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccc842-d555-404f-b7c5-9b85971457f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.460322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccc842-d555-404f-b7c5-9b85971457f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:47.462681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccc842-d555-404f-b7c5-9b85971457f2 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:47.477617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1136/2253] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:47.481781 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccc842-d555-404f-b7c5-9b85971457f2 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.492697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccc842-d555-404f-b7c5-9b85971457f2 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.497483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d815ad91-2e7a-4589-ba54-41518a3cca2c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.498519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d22309-ca7f-4701-b39d-9f9ff6956b98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.499351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d22309-ca7f-4701-b39d-9f9ff6956b98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.499640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d22309-ca7f-4701-b39d-9f9ff6956b98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.500011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d22309-ca7f-4701-b39d-9f9ff6956b98 admin admin] PATH_INFO: `/v3/projects/a286f61ad06343b0b213868853dfa111/users/647049618bab4749a78f161faa8c24f7/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.501493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d95e6-6859-4d03-9dc6-da0b1108875f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.502195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d95e6-6859-4d03-9dc6-da0b1108875f service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.502584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d95e6-6859-4d03-9dc6-da0b1108875f service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.502958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d95e6-6859-4d03-9dc6-da0b1108875f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.513226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d95e6-6859-4d03-9dc6-da0b1108875f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:47.515080 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d95e6-6859-4d03-9dc6-da0b1108875f service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:47.516753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d22309-ca7f-4701-b39d-9f9ff6956b98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a286f61ad06343b0b213868853dfa111, user_id=647049618bab4749a78f161faa8c24f7, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:47.519480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d22309-ca7f-4701-b39d-9f9ff6956b98 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:47.528427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1118/2254] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:44:47.535193 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1137/2255] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:47 2022] PUT /identity/v3/projects/a286f61ad06343b0b213868853dfa111/users/647049618bab4749a78f161faa8c24f7/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:44:47.539203 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d95e6-6859-4d03-9dc6-da0b1108875f service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.550926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d95e6-6859-4d03-9dc6-da0b1108875f service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.615209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee1d401-e1a0-40d3-9b0f-96693f41d904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.615821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1d401-e1a0-40d3-9b0f-96693f41d904 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.616101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1d401-e1a0-40d3-9b0f-96693f41d904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.616375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee1d401-e1a0-40d3-9b0f-96693f41d904 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.617054 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1d401-e1a0-40d3-9b0f-96693f41d904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:47.618706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ee1d401-e1a0-40d3-9b0f-96693f41d904 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:47.629336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1119/2256] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:47.634434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d22309-ca7f-4701-b39d-9f9ff6956b98 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.649204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d22309-ca7f-4701-b39d-9f9ff6956b98 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.658367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef382b4-c1bf-4336-881d-f0763434aee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.659503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef382b4-c1bf-4336-881d-f0763434aee7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.660050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef382b4-c1bf-4336-881d-f0763434aee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.660594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef382b4-c1bf-4336-881d-f0763434aee7 admin admin] PATH_INFO: `/v3/projects/a286f61ad06343b0b213868853dfa111/users/647049618bab4749a78f161faa8c24f7/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.664485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef382b4-c1bf-4336-881d-f0763434aee7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a286f61ad06343b0b213868853dfa111, user_id=647049618bab4749a78f161faa8c24f7, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:47.670873 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef382b4-c1bf-4336-881d-f0763434aee7 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:47.698587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1138/2257] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:47 2022] PUT /identity/v3/projects/a286f61ad06343b0b213868853dfa111/users/647049618bab4749a78f161faa8c24f7/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:47.711344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa60b-a905-42c9-a6a8-0fa64f689cce None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.711699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa60b-a905-42c9-a6a8-0fa64f689cce None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.712113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8aa60b-a905-42c9-a6a8-0fa64f689cce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.744170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2e8aa60b-a905-42c9-a6a8-0fa64f689cce None None] MFA Rules not processed for user `647049618bab4749a78f161faa8c24f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:47.746271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2e8aa60b-a905-42c9-a6a8-0fa64f689cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:47.822450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:47.850904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1120/2258] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:47.866306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef382b4-c1bf-4336-881d-f0763434aee7 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.868053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1ef382b4-c1bf-4336-881d-f0763434aee7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:47.948063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aa60b-a905-42c9-a6a8-0fa64f689cce None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:47.954133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2e8aa60b-a905-42c9-a6a8-0fa64f689cce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:47.976751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef382b4-c1bf-4336-881d-f0763434aee7 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:47.982923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d3095-b51d-4b43-af21-24c78d4b6379 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:47.983735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d3095-b51d-4b43-af21-24c78d4b6379 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:47.985315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d3095-b51d-4b43-af21-24c78d4b6379 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:47.985689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1d3095-b51d-4b43-af21-24c78d4b6379 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:47.996656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d3095-b51d-4b43-af21-24c78d4b6379 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:48.005679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd1d3095-b51d-4b43-af21-24c78d4b6379 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:48.016274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1139/2259] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:48.065570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d3095-b51d-4b43-af21-24c78d4b6379 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:48.067937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-cd1d3095-b51d-4b43-af21-24c78d4b6379 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:48.079491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1d3095-b51d-4b43-af21-24c78d4b6379 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:48.081355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8aa60b-a905-42c9-a6a8-0fa64f689cce None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:48.083203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d106ed-d079-42e6-8f63-fa5f5f144d51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:48.083805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d106ed-d079-42e6-8f63-fa5f5f144d51 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.084083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d106ed-d079-42e6-8f63-fa5f5f144d51 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.084355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d106ed-d079-42e6-8f63-fa5f5f144d51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.085971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:48.086123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-52d106ed-d079-42e6-8f63-fa5f5f144d51 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:48.086732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.087071 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.087479 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] PATH_INFO: `/v3/projects/7b93da1770e6470ea30f7d6661f7e96b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.088650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b93da1770e6470ea30f7d6661f7e96b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:48.090590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:48.092948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:48.139207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b93da1770e6470ea30f7d6661f7e96b', 'request_id': 'req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:48.145953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b93da1770e6470ea30f7d6661f7e96b', 'request_id': 'req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:48.150828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d106ed-d079-42e6-8f63-fa5f5f144d51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:48.152446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d106ed-d079-42e6-8f63-fa5f5f144d51 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:48.152984 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b93da1770e6470ea30f7d6661f7e96b', 'request_id': 'req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:48.162056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1140/2260] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:48 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:48.165938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] The token cache is being invalidate because project 7b93da1770e6470ea30f7d6661f7e96b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:48.165938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:48.166393 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fc4d9706-e8f7-4c27-90bf-a37e25d99a06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:48.168284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1121/2261] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:47 2022] DELETE /identity/v3/projects/7b93da1770e6470ea30f7d6661f7e96b => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:48.176440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1a176-4755-4098-aba1-b926d01fd8af None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.176731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1a176-4755-4098-aba1-b926d01fd8af None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.185098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e1a176-4755-4098-aba1-b926d01fd8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.208172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-55e1a176-4755-4098-aba1-b926d01fd8af None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:48.209430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-55e1a176-4755-4098-aba1-b926d01fd8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:48.274116 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1141/2262] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:48.282106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e7352-7266-4180-8509-f53abecabaeb None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.282531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e7352-7266-4180-8509-f53abecabaeb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.282956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3e7352-7266-4180-8509-f53abecabaeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.304339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-2b3e7352-7266-4180-8509-f53abecabaeb None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:48.305563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2b3e7352-7266-4180-8509-f53abecabaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:48.314000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1122/2263] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:48.318264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1a176-4755-4098-aba1-b926d01fd8af None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:48.327730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e1a176-4755-4098-aba1-b926d01fd8af None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:48.332856 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34fd66-fa2d-4a8e-858d-dc48c079f1a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:48.333457 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34fd66-fa2d-4a8e-858d-dc48c079f1a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.333729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34fd66-fa2d-4a8e-858d-dc48c079f1a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.333996 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c34fd66-fa2d-4a8e-858d-dc48c079f1a8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.334637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34fd66-fa2d-4a8e-858d-dc48c079f1a8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:48.334950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34fd66-fa2d-4a8e-858d-dc48c079f1a8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:48.336559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c34fd66-fa2d-4a8e-858d-dc48c079f1a8 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:48.350549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1142/2264] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:44:48.358057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e7352-7266-4180-8509-f53abecabaeb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:48.370036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3e7352-7266-4180-8509-f53abecabaeb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:48.377414 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd082124-3036-47ea-88eb-f2880cc03a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:48.378028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082124-3036-47ea-88eb-f2880cc03a26 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.378302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082124-3036-47ea-88eb-f2880cc03a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.378572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd082124-3036-47ea-88eb-f2880cc03a26 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.379755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd082124-3036-47ea-88eb-f2880cc03a26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:48.381738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd082124-3036-47ea-88eb-f2880cc03a26 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:48.414069 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1123/2265] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:48 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:48.418300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34fd66-fa2d-4a8e-858d-dc48c079f1a8 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:48.431335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c34fd66-fa2d-4a8e-858d-dc48c079f1a8 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:48.498812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eca73-d0d8-45e6-ab45-7c91f1c0cea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:48.499576 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eca73-d0d8-45e6-ab45-7c91f1c0cea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.500092 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eca73-d0d8-45e6-ab45-7c91f1c0cea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.500564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885eca73-d0d8-45e6-ab45-7c91f1c0cea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.501563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eca73-d0d8-45e6-ab45-7c91f1c0cea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:48.505383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885eca73-d0d8-45e6-ab45-7c91f1c0cea2 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:48.533583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1143/2266] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:48 2022] POST /identity/v3/users => generated 286 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:48.537255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd082124-3036-47ea-88eb-f2880cc03a26 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:48.550224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd082124-3036-47ea-88eb-f2880cc03a26 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:48.555537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0be752-2a70-43fe-9f19-1336679e876b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:48.556158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0be752-2a70-43fe-9f19-1336679e876b admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.556437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0be752-2a70-43fe-9f19-1336679e876b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.556700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0be752-2a70-43fe-9f19-1336679e876b admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.557448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0be752-2a70-43fe-9f19-1336679e876b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:48.559076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0be752-2a70-43fe-9f19-1336679e876b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:48.572941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1124/2267] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:48.577392 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eca73-d0d8-45e6-ab45-7c91f1c0cea2 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:48.596218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885eca73-d0d8-45e6-ab45-7c91f1c0cea2 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:48.601426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09807be7-e40c-4adc-be1d-4fe0ca28b261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:48.602143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09807be7-e40c-4adc-be1d-4fe0ca28b261 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.602429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09807be7-e40c-4adc-be1d-4fe0ca28b261 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.602704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09807be7-e40c-4adc-be1d-4fe0ca28b261 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.603347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09807be7-e40c-4adc-be1d-4fe0ca28b261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:48.604980 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09807be7-e40c-4adc-be1d-4fe0ca28b261 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:48.617166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1144/2268] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:48.621387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0be752-2a70-43fe-9f19-1336679e876b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:48.633751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0be752-2a70-43fe-9f19-1336679e876b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:48.638532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370b5e2-9240-40e2-bbb1-5273e88e7d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:48.639210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370b5e2-9240-40e2-bbb1-5273e88e7d37 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.639488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370b5e2-9240-40e2-bbb1-5273e88e7d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.639776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0370b5e2-9240-40e2-bbb1-5273e88e7d37 admin admin] PATH_INFO: `/v3/projects/6cc05f4c3f5442758ac3e0ee6c6167ea/users/db641ad103a54ee090f09a7ba6f8ea8a/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.655234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370b5e2-9240-40e2-bbb1-5273e88e7d37 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6cc05f4c3f5442758ac3e0ee6c6167ea, user_id=db641ad103a54ee090f09a7ba6f8ea8a, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:48.658643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0370b5e2-9240-40e2-bbb1-5273e88e7d37 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:48.670968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1125/2269] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:48 2022] PUT /identity/v3/projects/6cc05f4c3f5442758ac3e0ee6c6167ea/users/db641ad103a54ee090f09a7ba6f8ea8a/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:48.682867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b5e2b-806f-4fc9-aedc-50254c3b23af None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.683146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b5e2b-806f-4fc9-aedc-50254c3b23af None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.683417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b5e2b-806f-4fc9-aedc-50254c3b23af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.709704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:48.713101 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-577b5e2b-806f-4fc9-aedc-50254c3b23af None None] MFA Rules not processed for user `db641ad103a54ee090f09a7ba6f8ea8a`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:48.715188 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-577b5e2b-806f-4fc9-aedc-50254c3b23af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:48.785990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1145/2270] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:48.797664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370b5e2-9240-40e2-bbb1-5273e88e7d37 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:48.799495 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0370b5e2-9240-40e2-bbb1-5273e88e7d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:48.819793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:48.870710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0370b5e2-9240-40e2-bbb1-5273e88e7d37 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:48.875732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97dda689-86fc-4d9f-8338-71167a86017f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:48.876346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dda689-86fc-4d9f-8338-71167a86017f service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:48.876625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dda689-86fc-4d9f-8338-71167a86017f service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:48.876932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97dda689-86fc-4d9f-8338-71167a86017f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:48.885541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dda689-86fc-4d9f-8338-71167a86017f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:48.886965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97dda689-86fc-4d9f-8338-71167a86017f service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:48.897428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1126/2271] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:48.926645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b5e2b-806f-4fc9-aedc-50254c3b23af None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:49.014888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b5e2b-806f-4fc9-aedc-50254c3b23af None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:49.021352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:49.022173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:49.022599 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:49.022951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] PATH_INFO: `/v3/users/db641ad103a54ee090f09a7ba6f8ea8a` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:49.025007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db641ad103a54ee090f09a7ba6f8ea8a)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:49.027930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:49.081979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db641ad103a54ee090f09a7ba6f8ea8a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:49.090976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db641ad103a54ee090f09a7ba6f8ea8a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:49.092029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db641ad103a54ee090f09a7ba6f8ea8a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:49.106533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db641ad103a54ee090f09a7ba6f8ea8a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:49.114479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db641ad103a54ee090f09a7ba6f8ea8a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:49.120847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1146/2272] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:48 2022] DELETE /identity/v3/users/db641ad103a54ee090f09a7ba6f8ea8a => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:49.125225 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4a717-4aa5-45fa-98c0-21558f64d74c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:49.125506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4a717-4aa5-45fa-98c0-21558f64d74c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:49.125865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4a717-4aa5-45fa-98c0-21558f64d74c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:49.146731 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-53e4a717-4aa5-45fa-98c0-21558f64d74c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:49.147937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-53e4a717-4aa5-45fa-98c0-21558f64d74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:49.213499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1127/2273] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:49.223595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:49.225141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:49.252240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:49.290015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935db7dc-2c17-4ffb-a045-f9611dcb6f96 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:49.298167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b18bed-3c1d-4487-9602-65267580f175 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:49.298781 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b18bed-3c1d-4487-9602-65267580f175 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:49.299089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b18bed-3c1d-4487-9602-65267580f175 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:49.299364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b18bed-3c1d-4487-9602-65267580f175 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:49.313689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b18bed-3c1d-4487-9602-65267580f175 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:49.315134 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b18bed-3c1d-4487-9602-65267580f175 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:49.325521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1147/2274] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:49.605039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4a717-4aa5-45fa-98c0-21558f64d74c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:49.607144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-53e4a717-4aa5-45fa-98c0-21558f64d74c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:49.631129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4a717-4aa5-45fa-98c0-21558f64d74c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:49.646205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:49.647093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:49.647455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:49.647799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] PATH_INFO: `/v3/projects/6cc05f4c3f5442758ac3e0ee6c6167ea` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:49.649179 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6cc05f4c3f5442758ac3e0ee6c6167ea)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:49.651145 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:49.741395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6cc05f4c3f5442758ac3e0ee6c6167ea', 'request_id': 'req-202de0df-1d62-4203-af39-3ce156dc3593'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:49.753841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6cc05f4c3f5442758ac3e0ee6c6167ea', 'request_id': 'req-202de0df-1d62-4203-af39-3ce156dc3593'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:49.767357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6cc05f4c3f5442758ac3e0ee6c6167ea', 'request_id': 'req-202de0df-1d62-4203-af39-3ce156dc3593'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:49.779723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] The token cache is being invalidate because project 6cc05f4c3f5442758ac3e0ee6c6167ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:49.780092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:49.780816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-202de0df-1d62-4203-af39-3ce156dc3593 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:49.783435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1128/2275] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:49 2022] DELETE /identity/v3/projects/6cc05f4c3f5442758ac3e0ee6c6167ea => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:51.273821 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b18bed-3c1d-4487-9602-65267580f175 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:51.275210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e0b18bed-3c1d-4487-9602-65267580f175 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:51.290436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b18bed-3c1d-4487-9602-65267580f175 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:51.355307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:51.355935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:51.356211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:51.356479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] PATH_INFO: `/v3/users/b5643881ae9448a4be3575c202e310f6` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:51.357634 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5643881ae9448a4be3575c202e310f6)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:51.359604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:51.408194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5643881ae9448a4be3575c202e310f6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:51.414890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5643881ae9448a4be3575c202e310f6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:51.415680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5643881ae9448a4be3575c202e310f6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:51.427713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5643881ae9448a4be3575c202e310f6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:51.434474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5643881ae9448a4be3575c202e310f6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:51.440764 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1148/2276] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:51 2022] DELETE /identity/v3/users/b5643881ae9448a4be3575c202e310f6 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:44:51.445131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c657d8-b7f4-4676-ab2d-90110bb1a597 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:51.445583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c657d8-b7f4-4676-ab2d-90110bb1a597 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:51.445674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c657d8-b7f4-4676-ab2d-90110bb1a597 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:51.465298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-49c657d8-b7f4-4676-ab2d-90110bb1a597 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:51.466562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-49c657d8-b7f4-4676-ab2d-90110bb1a597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:51.509262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:51.532925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1129/2277] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:51.542540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:51.543930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:51.593072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:51.604957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f29f189-f8e9-4215-83ea-c8d3ded536cc admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:51.609717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d86182-692f-46d7-9d44-a41218f8b907 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:51.610629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d86182-692f-46d7-9d44-a41218f8b907 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:51.610940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d86182-692f-46d7-9d44-a41218f8b907 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:51.611174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d86182-692f-46d7-9d44-a41218f8b907 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:51.621358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d86182-692f-46d7-9d44-a41218f8b907 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:51.622779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d86182-692f-46d7-9d44-a41218f8b907 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:51.635275 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1149/2278] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:51.954689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c657d8-b7f4-4676-ab2d-90110bb1a597 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:51.956079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-49c657d8-b7f4-4676-ab2d-90110bb1a597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:51.969235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c657d8-b7f4-4676-ab2d-90110bb1a597 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:51.974032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:51.974683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:51.974956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:51.975224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] PATH_INFO: `/v3/projects/8eff3878504c40b08a176d29e30764bf` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:51.976357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8eff3878504c40b08a176d29e30764bf)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:51.978287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:52.024638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8eff3878504c40b08a176d29e30764bf', 'request_id': 'req-62a701f2-9611-4bf2-a200-a31e0254f651'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.032450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8eff3878504c40b08a176d29e30764bf', 'request_id': 'req-62a701f2-9611-4bf2-a200-a31e0254f651'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.040277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8eff3878504c40b08a176d29e30764bf', 'request_id': 'req-62a701f2-9611-4bf2-a200-a31e0254f651'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.063739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] The token cache is being invalidate because project 8eff3878504c40b08a176d29e30764bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:52.064154 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.065151 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.067092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1130/2279] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:51 2022] DELETE /identity/v3/projects/8eff3878504c40b08a176d29e30764bf => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:44:52.071417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d86182-692f-46d7-9d44-a41218f8b907 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:52.082624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d86182-692f-46d7-9d44-a41218f8b907 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:52.138835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:52.139586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:52.139697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:52.139889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] PATH_INFO: `/v3/users/42a117803c3142fd860c9fba32aca95b` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:52.141722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=42a117803c3142fd860c9fba32aca95b)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:52.143783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:52.192132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '42a117803c3142fd860c9fba32aca95b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.198374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '42a117803c3142fd860c9fba32aca95b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.198995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '42a117803c3142fd860c9fba32aca95b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.211368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '42a117803c3142fd860c9fba32aca95b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.218245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '42a117803c3142fd860c9fba32aca95b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.224221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1150/2280] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:52 2022] DELETE /identity/v3/users/42a117803c3142fd860c9fba32aca95b => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:44:52.516859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:52.518334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:52.532186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a701f2-9611-4bf2-a200-a31e0254f651 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:52.605824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:52.606581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:52.606957 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:52.607326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] PATH_INFO: `/v3/projects/7a1b2a6c1a4e4edaa1a40a3d424e75ba` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:52.608639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a1b2a6c1a4e4edaa1a40a3d424e75ba)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:52.611458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:52.656784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a1b2a6c1a4e4edaa1a40a3d424e75ba', 'request_id': 'req-c7452389-e3c6-482d-9c26-686348ff9267'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.662701 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a1b2a6c1a4e4edaa1a40a3d424e75ba', 'request_id': 'req-c7452389-e3c6-482d-9c26-686348ff9267'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.669718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a1b2a6c1a4e4edaa1a40a3d424e75ba', 'request_id': 'req-c7452389-e3c6-482d-9c26-686348ff9267'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.674848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:52.682277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] The token cache is being invalidate because project 7a1b2a6c1a4e4edaa1a40a3d424e75ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:52.682594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.683406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.685144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1131/2281] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:52 2022] DELETE /identity/v3/projects/7a1b2a6c1a4e4edaa1a40a3d424e75ba => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:52.689908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:52.691315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:52.704363 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e16adf-d7ea-42bc-bb39-bf32247af902 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:52.767834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b611c1-9d2d-4618-81d0-052d78ab16fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:52.768507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b611c1-9d2d-4618-81d0-052d78ab16fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:52.768749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b611c1-9d2d-4618-81d0-052d78ab16fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:52.769161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b611c1-9d2d-4618-81d0-052d78ab16fd admin admin] PATH_INFO: `/v3/users/647049618bab4749a78f161faa8c24f7` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:52.770418 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b611c1-9d2d-4618-81d0-052d78ab16fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=647049618bab4749a78f161faa8c24f7)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:52.772875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b611c1-9d2d-4618-81d0-052d78ab16fd admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:52.780430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:52.857413 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-77b611c1-9d2d-4618-81d0-052d78ab16fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '647049618bab4749a78f161faa8c24f7'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.865407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-77b611c1-9d2d-4618-81d0-052d78ab16fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '647049618bab4749a78f161faa8c24f7'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.866017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-77b611c1-9d2d-4618-81d0-052d78ab16fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '647049618bab4749a78f161faa8c24f7'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.879381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-77b611c1-9d2d-4618-81d0-052d78ab16fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '647049618bab4749a78f161faa8c24f7'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.887039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-77b611c1-9d2d-4618-81d0-052d78ab16fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '647049618bab4749a78f161faa8c24f7'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:52.894376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1151/2282] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:52 2022] DELETE /identity/v3/users/647049618bab4749a78f161faa8c24f7 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:53.214745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:53.216295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:53.233730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7452389-e3c6-482d-9c26-686348ff9267 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:53.300680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:53.301361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:53.301634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:53.301906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] PATH_INFO: `/v3/projects/a286f61ad06343b0b213868853dfa111` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:53.303042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] RBAC: Authorizing `identity:delete_project(project_id=a286f61ad06343b0b213868853dfa111)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:53.304983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:53.366855 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a286f61ad06343b0b213868853dfa111', 'request_id': 'req-2ed87d49-1055-4026-b76a-de073f5fe53e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:53.373487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a286f61ad06343b0b213868853dfa111', 'request_id': 'req-2ed87d49-1055-4026-b76a-de073f5fe53e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:53.381970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a286f61ad06343b0b213868853dfa111', 'request_id': 'req-2ed87d49-1055-4026-b76a-de073f5fe53e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:53.394385 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] The token cache is being invalidate because project a286f61ad06343b0b213868853dfa111 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:44:53.394668 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:53.395564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2ed87d49-1055-4026-b76a-de073f5fe53e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:53.397080 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1132/2283] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:44:53 2022] DELETE /identity/v3/projects/a286f61ad06343b0b213868853dfa111 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:53.402979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25e21f-67df-486f-a8ec-9856edf4e17c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:53.403339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25e21f-67df-486f-a8ec-9856edf4e17c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:53.403683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc25e21f-67df-486f-a8ec-9856edf4e17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:53.424315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-fc25e21f-67df-486f-a8ec-9856edf4e17c None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:53.425889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-fc25e21f-67df-486f-a8ec-9856edf4e17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:53.491125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1152/2284] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:53.500318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8c379-9ce2-41b9-8c65-8df3061ac39d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:53.500592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8c379-9ce2-41b9-8c65-8df3061ac39d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:53.500936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8c379-9ce2-41b9-8c65-8df3061ac39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:53.523105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-06a8c379-9ce2-41b9-8c65-8df3061ac39d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:53.524321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-06a8c379-9ce2-41b9-8c65-8df3061ac39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:53.533022 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1133/2285] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:53.537273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25e21f-67df-486f-a8ec-9856edf4e17c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:53.547744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc25e21f-67df-486f-a8ec-9856edf4e17c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:53.552259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4198d-a2f2-40c7-b843-298c1b120e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:53.553064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4198d-a2f2-40c7-b843-298c1b120e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:53.553334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4198d-a2f2-40c7-b843-298c1b120e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:53.553600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e4198d-a2f2-40c7-b843-298c1b120e7b admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:53.554245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4198d-a2f2-40c7-b843-298c1b120e7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:44:53.554565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4198d-a2f2-40c7-b843-298c1b120e7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:53.556504 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e4198d-a2f2-40c7-b843-298c1b120e7b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:53.569491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1153/2286] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:44:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:44:53.574274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8c379-9ce2-41b9-8c65-8df3061ac39d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:53.585285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8c379-9ce2-41b9-8c65-8df3061ac39d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:53.590712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01faf8-f1c9-4650-8960-f6fba8a8f7c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:53.591310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01faf8-f1c9-4650-8960-f6fba8a8f7c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:53.591586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01faf8-f1c9-4650-8960-f6fba8a8f7c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:53.591855 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef01faf8-f1c9-4650-8960-f6fba8a8f7c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:53.592478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01faf8-f1c9-4650-8960-f6fba8a8f7c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:53.594981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef01faf8-f1c9-4650-8960-f6fba8a8f7c4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:53.636450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1134/2287] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:53 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:53.638994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4198d-a2f2-40c7-b843-298c1b120e7b admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:53.651132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e4198d-a2f2-40c7-b843-298c1b120e7b admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:53.712431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f4893-1c1e-40a8-a0f9-76a57a10c5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:53.713181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f4893-1c1e-40a8-a0f9-76a57a10c5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:53.713341 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f4893-1c1e-40a8-a0f9-76a57a10c5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:53.713637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f4893-1c1e-40a8-a0f9-76a57a10c5b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:53.714415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f4893-1c1e-40a8-a0f9-76a57a10c5b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:53.717919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f4893-1c1e-40a8-a0f9-76a57a10c5b3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:53.743514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1154/2288] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:53 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:53.747637 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01faf8-f1c9-4650-8960-f6fba8a8f7c4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:53.759340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef01faf8-f1c9-4650-8960-f6fba8a8f7c4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:53.764188 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318f3b1-cd1e-4c9a-a90f-bd9d1fb7849c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:53.765321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318f3b1-cd1e-4c9a-a90f-bd9d1fb7849c admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:53.765321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318f3b1-cd1e-4c9a-a90f-bd9d1fb7849c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:53.766029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f318f3b1-cd1e-4c9a-a90f-bd9d1fb7849c admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:53.766658 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318f3b1-cd1e-4c9a-a90f-bd9d1fb7849c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:53.768261 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f318f3b1-cd1e-4c9a-a90f-bd9d1fb7849c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:53.778620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1135/2289] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:44:53.782758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f4893-1c1e-40a8-a0f9-76a57a10c5b3 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:53.789250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:53.793225 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f4893-1c1e-40a8-a0f9-76a57a10c5b3 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:53.808818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51480e19-8e71-4b6e-8da3-03a02e88089c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:53.809078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51480e19-8e71-4b6e-8da3-03a02e88089c admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:53.809228 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51480e19-8e71-4b6e-8da3-03a02e88089c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:53.809501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51480e19-8e71-4b6e-8da3-03a02e88089c admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:53.810225 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51480e19-8e71-4b6e-8da3-03a02e88089c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:53.811828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51480e19-8e71-4b6e-8da3-03a02e88089c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:53.823715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1155/2290] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:53.828538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318f3b1-cd1e-4c9a-a90f-bd9d1fb7849c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:53.838501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f318f3b1-cd1e-4c9a-a90f-bd9d1fb7849c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:53.843032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bb56a-eeaa-4df4-806f-acff0f17a330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:53.843669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bb56a-eeaa-4df4-806f-acff0f17a330 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:53.843940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bb56a-eeaa-4df4-806f-acff0f17a330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:53.844213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002bb56a-eeaa-4df4-806f-acff0f17a330 admin admin] PATH_INFO: `/v3/projects/1d7e0df76cd649deabced3b544aec4f3/users/35ae1cbace0c49209b0ac29f90c7434a/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:53.859086 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bb56a-eeaa-4df4-806f-acff0f17a330 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d7e0df76cd649deabced3b544aec4f3, user_id=35ae1cbace0c49209b0ac29f90c7434a, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:53.861732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002bb56a-eeaa-4df4-806f-acff0f17a330 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:53.876534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1136/2291] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:53 2022] PUT /identity/v3/projects/1d7e0df76cd649deabced3b544aec4f3/users/35ae1cbace0c49209b0ac29f90c7434a/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:53.888615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755baad-0d7a-456a-8897-84a24f98eb29 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:53.888936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755baad-0d7a-456a-8897-84a24f98eb29 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:53.889210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755baad-0d7a-456a-8897-84a24f98eb29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:53.918658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-a755baad-0d7a-456a-8897-84a24f98eb29 None None] MFA Rules not processed for user `35ae1cbace0c49209b0ac29f90c7434a`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:53.919883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a755baad-0d7a-456a-8897-84a24f98eb29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:53.982384 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1156/2292] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:53 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:53.987193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bb56a-eeaa-4df4-806f-acff0f17a330 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:54.015998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:54.058335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002bb56a-eeaa-4df4-806f-acff0f17a330 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:54.063636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eb2d8-3a3b-4b99-a9fa-394123af449c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:54.064237 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eb2d8-3a3b-4b99-a9fa-394123af449c admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:54.064525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eb2d8-3a3b-4b99-a9fa-394123af449c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:54.064798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6eb2d8-3a3b-4b99-a9fa-394123af449c admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:54.065427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6eb2d8-3a3b-4b99-a9fa-394123af449c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:54.067296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6eb2d8-3a3b-4b99-a9fa-394123af449c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:54.094232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1137/2293] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:44:53 2022] POST /identity/v3/projects => generated 365 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:44:54.099015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755baad-0d7a-456a-8897-84a24f98eb29 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:54.110449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755baad-0d7a-456a-8897-84a24f98eb29 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:54.194491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efa371-731c-48b5-845a-0b03082bf818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:54.195235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efa371-731c-48b5-845a-0b03082bf818 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:54.195595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efa371-731c-48b5-845a-0b03082bf818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:54.196067 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efa371-731c-48b5-845a-0b03082bf818 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:54.196813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efa371-731c-48b5-845a-0b03082bf818 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:54.199238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efa371-731c-48b5-845a-0b03082bf818 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:54.235287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1157/2294] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:44:54 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:44:54.239937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eb2d8-3a3b-4b99-a9fa-394123af449c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:54.252557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6eb2d8-3a3b-4b99-a9fa-394123af449c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:54.257456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815f703-6400-458e-aaf8-e80e6a488270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:54.258057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815f703-6400-458e-aaf8-e80e6a488270 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:54.258363 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815f703-6400-458e-aaf8-e80e6a488270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:54.258628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5815f703-6400-458e-aaf8-e80e6a488270 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:54.259274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815f703-6400-458e-aaf8-e80e6a488270 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:54.260882 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5815f703-6400-458e-aaf8-e80e6a488270 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:54.272959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1138/2295] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:54.277138 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efa371-731c-48b5-845a-0b03082bf818 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:54.288965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efa371-731c-48b5-845a-0b03082bf818 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:54.295322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9853c-4b15-437c-9c3b-76da53989874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:54.296121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9853c-4b15-437c-9c3b-76da53989874 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:54.296495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9853c-4b15-437c-9c3b-76da53989874 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:54.297405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e9853c-4b15-437c-9c3b-76da53989874 admin admin] PATH_INFO: `/v3/projects/e095fcf242ba4a89ba6cdf78f1e53fa9/users/a62c9eb06ba24ef9b7bc6b1557484992/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:54.317572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9853c-4b15-437c-9c3b-76da53989874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e095fcf242ba4a89ba6cdf78f1e53fa9, user_id=a62c9eb06ba24ef9b7bc6b1557484992, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:54.321020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e9853c-4b15-437c-9c3b-76da53989874 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:54.340152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1158/2296] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:54 2022] PUT /identity/v3/projects/e095fcf242ba4a89ba6cdf78f1e53fa9/users/a62c9eb06ba24ef9b7bc6b1557484992/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:54.344868 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815f703-6400-458e-aaf8-e80e6a488270 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:54.355322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5815f703-6400-458e-aaf8-e80e6a488270 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:54.423377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927fe014-60ae-4e4c-b3e7-1a8110864bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:54.423993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fe014-60ae-4e4c-b3e7-1a8110864bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:54.424271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fe014-60ae-4e4c-b3e7-1a8110864bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:54.424541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927fe014-60ae-4e4c-b3e7-1a8110864bd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:54.425234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fe014-60ae-4e4c-b3e7-1a8110864bd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:54.426832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927fe014-60ae-4e4c-b3e7-1a8110864bd4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:54.438424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1139/2297] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:44:54 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:44:54.442488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9853c-4b15-437c-9c3b-76da53989874 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:54.453268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e9853c-4b15-437c-9c3b-76da53989874 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:54.458043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21455314-8a89-498a-84e6-6dc2b384a571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:54.458738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21455314-8a89-498a-84e6-6dc2b384a571 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:54.459010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21455314-8a89-498a-84e6-6dc2b384a571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:54.459277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21455314-8a89-498a-84e6-6dc2b384a571 admin admin] PATH_INFO: `/v3/projects/e095fcf242ba4a89ba6cdf78f1e53fa9/users/a62c9eb06ba24ef9b7bc6b1557484992/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:54.461258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21455314-8a89-498a-84e6-6dc2b384a571 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e095fcf242ba4a89ba6cdf78f1e53fa9, user_id=a62c9eb06ba24ef9b7bc6b1557484992, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:54.463994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21455314-8a89-498a-84e6-6dc2b384a571 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:54.475847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1159/2298] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:44:54 2022] PUT /identity/v3/projects/e095fcf242ba4a89ba6cdf78f1e53fa9/users/a62c9eb06ba24ef9b7bc6b1557484992/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:44:54.487310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7429091-1a64-4fda-a489-55f47d5bb375 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:54.487590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7429091-1a64-4fda-a489-55f47d5bb375 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:54.487863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7429091-1a64-4fda-a489-55f47d5bb375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:54.516872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e7429091-1a64-4fda-a489-55f47d5bb375 None None] MFA Rules not processed for user `a62c9eb06ba24ef9b7bc6b1557484992`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:44:54.518108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e7429091-1a64-4fda-a489-55f47d5bb375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:54.599423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1140/2299] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:44:54 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:44:54.610936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21455314-8a89-498a-84e6-6dc2b384a571 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:54.612744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-21455314-8a89-498a-84e6-6dc2b384a571 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:44:54.650854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:54.680945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21455314-8a89-498a-84e6-6dc2b384a571 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:54.684516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb4ba1-fd7a-489e-8b1e-91eff383972f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:54.685149 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb4ba1-fd7a-489e-8b1e-91eff383972f service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:54.685420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb4ba1-fd7a-489e-8b1e-91eff383972f service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:54.685685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fb4ba1-fd7a-489e-8b1e-91eff383972f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:54.694688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb4ba1-fd7a-489e-8b1e-91eff383972f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:54.696088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fb4ba1-fd7a-489e-8b1e-91eff383972f service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:54.705600 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1160/2300] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:54 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:54.794583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7429091-1a64-4fda-a489-55f47d5bb375 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:54.805294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7429091-1a64-4fda-a489-55f47d5bb375 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:54.809104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799ef154-d99c-4010-8c8d-efca17524f22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:54.809730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ef154-d99c-4010-8c8d-efca17524f22 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:54.810057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ef154-d99c-4010-8c8d-efca17524f22 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:54.810340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799ef154-d99c-4010-8c8d-efca17524f22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:54.823737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:44:54.877447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ef154-d99c-4010-8c8d-efca17524f22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:54.879129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799ef154-d99c-4010-8c8d-efca17524f22 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:54.892501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1141/2301] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:44:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:44:59.852132 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb4ba1-fd7a-489e-8b1e-91eff383972f service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:44:59.913653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fb4ba1-fd7a-489e-8b1e-91eff383972f service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:44:59.918184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:44:59.918808 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:44:59.919078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:44:59.919370 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] PATH_INFO: `/v3/users/35ae1cbace0c49209b0ac29f90c7434a` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:44:59.920458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=35ae1cbace0c49209b0ac29f90c7434a)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:44:59.922422 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:44:59.970078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35ae1cbace0c49209b0ac29f90c7434a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:59.978006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35ae1cbace0c49209b0ac29f90c7434a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:59.978731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35ae1cbace0c49209b0ac29f90c7434a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:59.990792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35ae1cbace0c49209b0ac29f90c7434a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:44:59.997500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35ae1cbace0c49209b0ac29f90c7434a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:00.003773 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1161/2302] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:44:59 2022] DELETE /identity/v3/users/35ae1cbace0c49209b0ac29f90c7434a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:00.008364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3015184-26ea-4fc0-972b-e67a21c52a13 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:00.008867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3015184-26ea-4fc0-972b-e67a21c52a13 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:00.008994 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3015184-26ea-4fc0-972b-e67a21c52a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:00.028174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a3015184-26ea-4fc0-972b-e67a21c52a13 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:00.029457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a3015184-26ea-4fc0-972b-e67a21c52a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:00.087799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1142/2303] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:00.097983 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:00.099395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:00.156638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b823d5-3896-4966-b08a-9232fb0417e0 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:00.160163 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2871b-9ac4-421e-a16d-7072855333f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:00.160773 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2871b-9ac4-421e-a16d-7072855333f3 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:00.161055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2871b-9ac4-421e-a16d-7072855333f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:00.161319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e2871b-9ac4-421e-a16d-7072855333f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:00.169995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2871b-9ac4-421e-a16d-7072855333f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:00.171393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e2871b-9ac4-421e-a16d-7072855333f3 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:00.175327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:00.188341 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1162/2304] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:45:00.461528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3015184-26ea-4fc0-972b-e67a21c52a13 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:00.463051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a3015184-26ea-4fc0-972b-e67a21c52a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:00.476032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3015184-26ea-4fc0-972b-e67a21c52a13 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:00.488411 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:00.489115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:00.489444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:00.489724 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] PATH_INFO: `/v3/projects/1d7e0df76cd649deabced3b544aec4f3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:00.490917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d7e0df76cd649deabced3b544aec4f3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:00.493556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:00.543504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d7e0df76cd649deabced3b544aec4f3', 'request_id': 'req-0cb0537b-3e57-4665-8dad-340973dbce77'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:00.554313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d7e0df76cd649deabced3b544aec4f3', 'request_id': 'req-0cb0537b-3e57-4665-8dad-340973dbce77'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:00.563758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d7e0df76cd649deabced3b544aec4f3', 'request_id': 'req-0cb0537b-3e57-4665-8dad-340973dbce77'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:00.583891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] The token cache is being invalidate because project 1d7e0df76cd649deabced3b544aec4f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:45:00.584397 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:00.585202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:00.587646 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1143/2305] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:45:00 2022] DELETE /identity/v3/projects/1d7e0df76cd649deabced3b544aec4f3 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:00.591480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2871b-9ac4-421e-a16d-7072855333f3 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:00.601773 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e2871b-9ac4-421e-a16d-7072855333f3 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:00.665553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1049a2a1-e8d7-4a64-a798-f4997a266593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:00.666473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a2a1-e8d7-4a64-a798-f4997a266593 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:00.666752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a2a1-e8d7-4a64-a798-f4997a266593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:00.667094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1049a2a1-e8d7-4a64-a798-f4997a266593 admin admin] PATH_INFO: `/v3/users/a62c9eb06ba24ef9b7bc6b1557484992` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:00.668361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049a2a1-e8d7-4a64-a798-f4997a266593 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a62c9eb06ba24ef9b7bc6b1557484992)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:00.670411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1049a2a1-e8d7-4a64-a798-f4997a266593 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:00.729926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-1049a2a1-e8d7-4a64-a798-f4997a266593 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a62c9eb06ba24ef9b7bc6b1557484992'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:00.737475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-1049a2a1-e8d7-4a64-a798-f4997a266593 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a62c9eb06ba24ef9b7bc6b1557484992'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:00.738289 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-1049a2a1-e8d7-4a64-a798-f4997a266593 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a62c9eb06ba24ef9b7bc6b1557484992'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:00.750837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-1049a2a1-e8d7-4a64-a798-f4997a266593 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a62c9eb06ba24ef9b7bc6b1557484992'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:00.757837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-1049a2a1-e8d7-4a64-a798-f4997a266593 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a62c9eb06ba24ef9b7bc6b1557484992'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:00.763840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1163/2306] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:45:00 2022] DELETE /identity/v3/users/a62c9eb06ba24ef9b7bc6b1557484992 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:01.037212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:01.038698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:01.063590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb0537b-3e57-4665-8dad-340973dbce77 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:01.102820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:01.122321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:01.122946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:01.123216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:01.123480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] PATH_INFO: `/v3/projects/e095fcf242ba4a89ba6cdf78f1e53fa9` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:01.124634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e095fcf242ba4a89ba6cdf78f1e53fa9)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:01.127173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:01.173276 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e095fcf242ba4a89ba6cdf78f1e53fa9', 'request_id': 'req-f45c5857-f37e-4c0f-9e90-36b52004b362'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:01.179275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e095fcf242ba4a89ba6cdf78f1e53fa9', 'request_id': 'req-f45c5857-f37e-4c0f-9e90-36b52004b362'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:01.186402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e095fcf242ba4a89ba6cdf78f1e53fa9', 'request_id': 'req-f45c5857-f37e-4c0f-9e90-36b52004b362'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:01.197476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] The token cache is being invalidate because project e095fcf242ba4a89ba6cdf78f1e53fa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:45:01.197762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:01.198377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f45c5857-f37e-4c0f-9e90-36b52004b362 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:01.199845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1144/2307] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:45:01 2022] DELETE /identity/v3/projects/e095fcf242ba4a89ba6cdf78f1e53fa9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:01.204855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e45cf-39f2-4144-9811-9618539d6b57 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:01.205332 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e45cf-39f2-4144-9811-9618539d6b57 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:01.205484 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977e45cf-39f2-4144-9811-9618539d6b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:01.225697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-977e45cf-39f2-4144-9811-9618539d6b57 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:01.226846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-977e45cf-39f2-4144-9811-9618539d6b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:01.285548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1164/2308] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:01.293034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4795f-522e-47b7-9ecd-8009d09feecc None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:01.293316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4795f-522e-47b7-9ecd-8009d09feecc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:01.293587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a4795f-522e-47b7-9ecd-8009d09feecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:01.313918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-42a4795f-522e-47b7-9ecd-8009d09feecc None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:01.315094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-42a4795f-522e-47b7-9ecd-8009d09feecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:01.322814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1145/2309] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:01.326971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e45cf-39f2-4144-9811-9618539d6b57 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:01.336611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977e45cf-39f2-4144-9811-9618539d6b57 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:01.341287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851fcca-b632-4866-a1f6-7691fe2178d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:01.341903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851fcca-b632-4866-a1f6-7691fe2178d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:01.342179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851fcca-b632-4866-a1f6-7691fe2178d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:01.342446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5851fcca-b632-4866-a1f6-7691fe2178d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:01.343085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851fcca-b632-4866-a1f6-7691fe2178d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:45:01.343877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851fcca-b632-4866-a1f6-7691fe2178d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:01.345596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5851fcca-b632-4866-a1f6-7691fe2178d5 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:01.357534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1165/2310] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:45:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:45:01.362607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4795f-522e-47b7-9ecd-8009d09feecc None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:01.372001 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a4795f-522e-47b7-9ecd-8009d09feecc None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:01.376948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec38fe-2c8c-4c1d-ae42-d82f9686ec1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:01.377537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec38fe-2c8c-4c1d-ae42-d82f9686ec1f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:01.377809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec38fe-2c8c-4c1d-ae42-d82f9686ec1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:01.378072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ec38fe-2c8c-4c1d-ae42-d82f9686ec1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:01.378696 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec38fe-2c8c-4c1d-ae42-d82f9686ec1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:01.380571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ec38fe-2c8c-4c1d-ae42-d82f9686ec1f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:01.407175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1146/2311] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:45:01 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:45:01.411827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851fcca-b632-4866-a1f6-7691fe2178d5 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:01.418178 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:01.422903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5851fcca-b632-4866-a1f6-7691fe2178d5 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:01.484989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecce40-893f-43fb-be7d-66a11184e34a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:01.485588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecce40-893f-43fb-be7d-66a11184e34a admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:01.485869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecce40-893f-43fb-be7d-66a11184e34a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:01.486173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecce40-893f-43fb-be7d-66a11184e34a admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:01.486771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecce40-893f-43fb-be7d-66a11184e34a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:01.488793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecce40-893f-43fb-be7d-66a11184e34a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:01.513326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1166/2312] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:45:01 2022] POST /identity/v3/users => generated 290 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:45:01.517632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec38fe-2c8c-4c1d-ae42-d82f9686ec1f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:01.528368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ec38fe-2c8c-4c1d-ae42-d82f9686ec1f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:01.534212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da6204-eb01-4944-83ba-2745ffba0774 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:01.534793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da6204-eb01-4944-83ba-2745ffba0774 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:01.535044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da6204-eb01-4944-83ba-2745ffba0774 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:01.535312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4da6204-eb01-4944-83ba-2745ffba0774 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:01.535943 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da6204-eb01-4944-83ba-2745ffba0774 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:01.537550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4da6204-eb01-4944-83ba-2745ffba0774 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:01.549011 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1147/2313] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:45:01.553152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecce40-893f-43fb-be7d-66a11184e34a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:01.566090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecce40-893f-43fb-be7d-66a11184e34a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:01.571253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86241dc-0046-49ba-addc-3a378f535afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:01.571868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86241dc-0046-49ba-addc-3a378f535afd admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:01.572317 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86241dc-0046-49ba-addc-3a378f535afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:01.572596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86241dc-0046-49ba-addc-3a378f535afd admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:01.573278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86241dc-0046-49ba-addc-3a378f535afd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:01.575270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86241dc-0046-49ba-addc-3a378f535afd admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:01.588328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1167/2314] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:01 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:45:01.592017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da6204-eb01-4944-83ba-2745ffba0774 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:01.602843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4da6204-eb01-4944-83ba-2745ffba0774 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:01.607530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9441a-4c1f-4016-bbba-2105f5de18f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:01.608242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9441a-4c1f-4016-bbba-2105f5de18f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:01.608599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9441a-4c1f-4016-bbba-2105f5de18f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:01.609604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a9441a-4c1f-4016-bbba-2105f5de18f1 admin admin] PATH_INFO: `/v3/projects/3008540c9a114192bb6ec5a2771c67f0/users/1c0b5e3ae8254d72bf3b78856d738267/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:01.626428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9441a-4c1f-4016-bbba-2105f5de18f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3008540c9a114192bb6ec5a2771c67f0, user_id=1c0b5e3ae8254d72bf3b78856d738267, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:01.629241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a9441a-4c1f-4016-bbba-2105f5de18f1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:01.643733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1148/2315] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:45:01 2022] PUT /identity/v3/projects/3008540c9a114192bb6ec5a2771c67f0/users/1c0b5e3ae8254d72bf3b78856d738267/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:01.655094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f324967-7d81-4f5d-b804-34cadb2319f6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:01.655379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f324967-7d81-4f5d-b804-34cadb2319f6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:01.655654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f324967-7d81-4f5d-b804-34cadb2319f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:01.697500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2f324967-7d81-4f5d-b804-34cadb2319f6 None None] MFA Rules not processed for user `1c0b5e3ae8254d72bf3b78856d738267`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:01.698747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2f324967-7d81-4f5d-b804-34cadb2319f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:01.817674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1168/2316] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:01 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:01.817949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9441a-4c1f-4016-bbba-2105f5de18f1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:01.899411 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a9441a-4c1f-4016-bbba-2105f5de18f1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:01.904298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35025e30-03ce-47d4-add1-1153c7092f77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:01.904935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35025e30-03ce-47d4-add1-1153c7092f77 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:01.905213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35025e30-03ce-47d4-add1-1153c7092f77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:01.905483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35025e30-03ce-47d4-add1-1153c7092f77 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:01.906207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35025e30-03ce-47d4-add1-1153c7092f77 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:01.908109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35025e30-03ce-47d4-add1-1153c7092f77 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:01.920127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:01.942331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1149/2317] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:45:01 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:45:01.947405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f324967-7d81-4f5d-b804-34cadb2319f6 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:01.959081 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f324967-7d81-4f5d-b804-34cadb2319f6 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:02.029649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a56690-08b3-4fcf-8753-444d93b73cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:02.030506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a56690-08b3-4fcf-8753-444d93b73cd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:02.030821 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a56690-08b3-4fcf-8753-444d93b73cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:02.031144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a56690-08b3-4fcf-8753-444d93b73cd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:02.031896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a56690-08b3-4fcf-8753-444d93b73cd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:02.033862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a56690-08b3-4fcf-8753-444d93b73cd3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:02.066029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1169/2318] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:45:01 2022] POST /identity/v3/users => generated 297 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:45:02.069814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35025e30-03ce-47d4-add1-1153c7092f77 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:02.083340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35025e30-03ce-47d4-add1-1153c7092f77 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:02.089162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902cfbe-287b-4227-bceb-d398d90a760e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:02.090043 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902cfbe-287b-4227-bceb-d398d90a760e admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:02.090358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902cfbe-287b-4227-bceb-d398d90a760e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:02.090682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0902cfbe-287b-4227-bceb-d398d90a760e admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:02.091320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902cfbe-287b-4227-bceb-d398d90a760e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:02.093180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0902cfbe-287b-4227-bceb-d398d90a760e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:02.104497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1150/2319] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:45:02.108917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a56690-08b3-4fcf-8753-444d93b73cd3 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:02.126935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:02.132850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a56690-08b3-4fcf-8753-444d93b73cd3 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:02.140871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41060255-4699-4196-bacc-a9e3bf720757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:02.141706 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41060255-4699-4196-bacc-a9e3bf720757 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:02.141883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41060255-4699-4196-bacc-a9e3bf720757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:02.142510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41060255-4699-4196-bacc-a9e3bf720757 admin admin] PATH_INFO: `/v3/projects/781a06c93eb84d1780f8a7febb33fb9e/users/9f8cbc5bd72c49999a89c7ca28bda01c/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:02.161436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41060255-4699-4196-bacc-a9e3bf720757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=781a06c93eb84d1780f8a7febb33fb9e, user_id=9f8cbc5bd72c49999a89c7ca28bda01c, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:02.164505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41060255-4699-4196-bacc-a9e3bf720757 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:02.178716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1170/2320] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:45:02 2022] PUT /identity/v3/projects/781a06c93eb84d1780f8a7febb33fb9e/users/9f8cbc5bd72c49999a89c7ca28bda01c/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:02.182754 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902cfbe-287b-4227-bceb-d398d90a760e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:02.193564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0902cfbe-287b-4227-bceb-d398d90a760e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:02.266112 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb2181-7edc-49b5-b317-6ca5d7af7d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:02.266723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb2181-7edc-49b5-b317-6ca5d7af7d9f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:02.267004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb2181-7edc-49b5-b317-6ca5d7af7d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:02.267268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb2181-7edc-49b5-b317-6ca5d7af7d9f admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:02.267911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb2181-7edc-49b5-b317-6ca5d7af7d9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:02.269542 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb2181-7edc-49b5-b317-6ca5d7af7d9f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:02.280524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1151/2321] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:02 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:45:02.284747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41060255-4699-4196-bacc-a9e3bf720757 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:02.295079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41060255-4699-4196-bacc-a9e3bf720757 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:02.299911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2486f3-1511-4d4f-9437-3d388907768a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:02.300561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2486f3-1511-4d4f-9437-3d388907768a admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:02.300865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2486f3-1511-4d4f-9437-3d388907768a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:02.301161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2486f3-1511-4d4f-9437-3d388907768a admin admin] PATH_INFO: `/v3/projects/781a06c93eb84d1780f8a7febb33fb9e/users/9f8cbc5bd72c49999a89c7ca28bda01c/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:02.303068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2486f3-1511-4d4f-9437-3d388907768a admin admin] RBAC: Authorizing `identity:create_grant(project_id=781a06c93eb84d1780f8a7febb33fb9e, user_id=9f8cbc5bd72c49999a89c7ca28bda01c, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:02.305724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2486f3-1511-4d4f-9437-3d388907768a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:02.318455 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1171/2322] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:45:02 2022] PUT /identity/v3/projects/781a06c93eb84d1780f8a7febb33fb9e/users/9f8cbc5bd72c49999a89c7ca28bda01c/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:02.333029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01827e9-054a-486e-850e-0e92131725e3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:02.333345 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01827e9-054a-486e-850e-0e92131725e3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:02.333680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01827e9-054a-486e-850e-0e92131725e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:02.366115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a01827e9-054a-486e-850e-0e92131725e3 None None] MFA Rules not processed for user `9f8cbc5bd72c49999a89c7ca28bda01c`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:02.368136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a01827e9-054a-486e-850e-0e92131725e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:02.445669 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1152/2323] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:02 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:02.458838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2486f3-1511-4d4f-9437-3d388907768a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:02.460516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4c2486f3-1511-4d4f-9437-3d388907768a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:02.521043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2486f3-1511-4d4f-9437-3d388907768a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:02.524649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e52f413-ed2b-4d97-84c7-137f4763d538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:02.525287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52f413-ed2b-4d97-84c7-137f4763d538 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:02.525561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52f413-ed2b-4d97-84c7-137f4763d538 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:02.525833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e52f413-ed2b-4d97-84c7-137f4763d538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:02.535513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52f413-ed2b-4d97-84c7-137f4763d538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:02.536962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e52f413-ed2b-4d97-84c7-137f4763d538 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:02.546345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1172/2324] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:02 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:45:02.961846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01827e9-054a-486e-850e-0e92131725e3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:03.051555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01827e9-054a-486e-850e-0e92131725e3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:03.056567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:03.057305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:03.057583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:03.057861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] PATH_INFO: `/v3/users/1c0b5e3ae8254d72bf3b78856d738267` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:03.058982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c0b5e3ae8254d72bf3b78856d738267)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:03.061448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:03.064657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:03.122876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c0b5e3ae8254d72bf3b78856d738267'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:03.123973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c0b5e3ae8254d72bf3b78856d738267'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:03.134599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c0b5e3ae8254d72bf3b78856d738267'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:03.148357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c0b5e3ae8254d72bf3b78856d738267'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:03.167571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c0b5e3ae8254d72bf3b78856d738267'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:03.176232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1153/2325] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:45:02 2022] DELETE /identity/v3/users/1c0b5e3ae8254d72bf3b78856d738267 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:03.181030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f063c-0a97-4aea-a78f-c796fddd2f92 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:03.181302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f063c-0a97-4aea-a78f-c796fddd2f92 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:03.181577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f063c-0a97-4aea-a78f-c796fddd2f92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:03.203650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-791f063c-0a97-4aea-a78f-c796fddd2f92 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:03.204923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-791f063c-0a97-4aea-a78f-c796fddd2f92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:03.293509 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1173/2326] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:03.305555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:03.307102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:03.389218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7aba2d-5897-47a9-ace9-601227784ecd admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:03.394681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc77d0-9b92-4adf-9064-025ba93edefe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:03.395887 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc77d0-9b92-4adf-9064-025ba93edefe service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:03.396282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc77d0-9b92-4adf-9064-025ba93edefe service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:03.396726 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc77d0-9b92-4adf-9064-025ba93edefe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:03.407584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc77d0-9b92-4adf-9064-025ba93edefe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:03.410043 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc77d0-9b92-4adf-9064-025ba93edefe service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:03.422733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1154/2327] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:45:03.489623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d28fb-b7a0-486d-b5a6-4ee44b7f9757 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:03.490004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d28fb-b7a0-486d-b5a6-4ee44b7f9757 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:03.490486 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069d28fb-b7a0-486d-b5a6-4ee44b7f9757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:03.512119 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-069d28fb-b7a0-486d-b5a6-4ee44b7f9757 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:03.513637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-069d28fb-b7a0-486d-b5a6-4ee44b7f9757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:03.605886 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1174/2328] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:45:03 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:03.664128 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc77d0-9b92-4adf-9064-025ba93edefe service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:03.666430 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-9fcc77d0-9b92-4adf-9064-025ba93edefe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:03.772651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc77d0-9b92-4adf-9064-025ba93edefe service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:03.782152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14550ccd-74e6-4a06-b682-ff254d465f0c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:03.784610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14550ccd-74e6-4a06-b682-ff254d465f0c service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:03.784751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14550ccd-74e6-4a06-b682-ff254d465f0c service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:03.785268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14550ccd-74e6-4a06-b682-ff254d465f0c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:03.800595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14550ccd-74e6-4a06-b682-ff254d465f0c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:03.803869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14550ccd-74e6-4a06-b682-ff254d465f0c service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:03.820114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1155/2329] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:45:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:45:03.876430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d28fb-b7a0-486d-b5a6-4ee44b7f9757 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:03.878785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-069d28fb-b7a0-486d-b5a6-4ee44b7f9757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:03.900594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069d28fb-b7a0-486d-b5a6-4ee44b7f9757 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:03.907124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:03.907770 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:03.908068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:03.908460 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] PATH_INFO: `/v3/projects/3008540c9a114192bb6ec5a2771c67f0` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:03.909843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3008540c9a114192bb6ec5a2771c67f0)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:03.911993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:03.921229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:03.990969 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3008540c9a114192bb6ec5a2771c67f0', 'request_id': 'req-2c291baa-6ad5-4ea3-960e-67d55407db93'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.001021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3008540c9a114192bb6ec5a2771c67f0', 'request_id': 'req-2c291baa-6ad5-4ea3-960e-67d55407db93'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.010710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3008540c9a114192bb6ec5a2771c67f0', 'request_id': 'req-2c291baa-6ad5-4ea3-960e-67d55407db93'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.019850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] The token cache is being invalidate because project 3008540c9a114192bb6ec5a2771c67f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:45:04.020296 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.021053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2c291baa-6ad5-4ea3-960e-67d55407db93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.022968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1175/2330] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:45:03 2022] DELETE /identity/v3/projects/3008540c9a114192bb6ec5a2771c67f0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:04.029175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14550ccd-74e6-4a06-b682-ff254d465f0c service swift] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:04.042597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14550ccd-74e6-4a06-b682-ff254d465f0c service swift] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:04.059213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:04.136253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:04.136928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:04.137204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:04.138154 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] PATH_INFO: `/v3/users/9f8cbc5bd72c49999a89c7ca28bda01c` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:04.138553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f8cbc5bd72c49999a89c7ca28bda01c)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:04.140714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:04.200990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f8cbc5bd72c49999a89c7ca28bda01c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.201952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f8cbc5bd72c49999a89c7ca28bda01c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.215670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f8cbc5bd72c49999a89c7ca28bda01c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.226948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f8cbc5bd72c49999a89c7ca28bda01c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.243626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f8cbc5bd72c49999a89c7ca28bda01c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.252075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1156/2331] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:45:04 2022] DELETE /identity/v3/users/9f8cbc5bd72c49999a89c7ca28bda01c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:04.542605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58e34-2bb3-4143-bdc6-d87ba7ace779 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:04.543566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58e34-2bb3-4143-bdc6-d87ba7ace779 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:04.544088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d58e34-2bb3-4143-bdc6-d87ba7ace779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:04.566787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d7d58e34-2bb3-4143-bdc6-d87ba7ace779 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:04.568117 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d7d58e34-2bb3-4143-bdc6-d87ba7ace779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:04.611290 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:04.612963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:04.677655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1176/2332] 10.222.0.39 () {58 vars in 967 bytes} [Tue Sep 6 22:45:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:45:04.692877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d58e34-2bb3-4143-bdc6-d87ba7ace779 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:04.694800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d7d58e34-2bb3-4143-bdc6-d87ba7ace779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:04.735273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293dd6ad-c715-4a9e-b19b-279961707401 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:04.740155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:04.740841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:04.741109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:04.741535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] PATH_INFO: `/v3/projects/781a06c93eb84d1780f8a7febb33fb9e` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:04.742674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=781a06c93eb84d1780f8a7febb33fb9e)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:04.744579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:04.783906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d58e34-2bb3-4143-bdc6-d87ba7ace779 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:04.789042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:04.804244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '781a06c93eb84d1780f8a7febb33fb9e', 'request_id': 'req-baeb4d08-d937-4410-af44-d7ca9bd369bd'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.813983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '781a06c93eb84d1780f8a7febb33fb9e', 'request_id': 'req-baeb4d08-d937-4410-af44-d7ca9bd369bd'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.824634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '781a06c93eb84d1780f8a7febb33fb9e', 'request_id': 'req-baeb4d08-d937-4410-af44-d7ca9bd369bd'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.833364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:04.837169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] The token cache is being invalidate because project 781a06c93eb84d1780f8a7febb33fb9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:45:04.837461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.838120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-baeb4d08-d937-4410-af44-d7ca9bd369bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:04.840099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1157/2333] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:45:04 2022] DELETE /identity/v3/projects/781a06c93eb84d1780f8a7febb33fb9e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:04.845234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4ea03-2f14-4686-ad62-e531690a4f75 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:04.845587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4ea03-2f14-4686-ad62-e531690a4f75 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:04.845866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4ea03-2f14-4686-ad62-e531690a4f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:04.863085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70601a46-bfb5-4951-8bd6-2abda7d69594 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:04.863706 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70601a46-bfb5-4951-8bd6-2abda7d69594 service swift] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:04.864009 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70601a46-bfb5-4951-8bd6-2abda7d69594 service swift] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:04.864289 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70601a46-bfb5-4951-8bd6-2abda7d69594 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:04.866315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-70601a46-bfb5-4951-8bd6-2abda7d69594 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:04.868836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8cc4ea03-2f14-4686-ad62-e531690a4f75 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:04.870172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8cc4ea03-2f14-4686-ad62-e531690a4f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:04.945371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70601a46-bfb5-4951-8bd6-2abda7d69594 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:04.947142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70601a46-bfb5-4951-8bd6-2abda7d69594 service swift] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:04.957359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1177/2334] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:45:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 22:45:04.963362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1158/2335] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:45:04.973695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3c303-3b3c-452e-99cd-4e92ab369960 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:04.973993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3c303-3b3c-452e-99cd-4e92ab369960 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:04.974274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3c303-3b3c-452e-99cd-4e92ab369960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:04.998749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-5ba3c303-3b3c-452e-99cd-4e92ab369960 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:05.000948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5ba3c303-3b3c-452e-99cd-4e92ab369960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:05.010962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1178/2336] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:05.016809 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4ea03-2f14-4686-ad62-e531690a4f75 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:05.028118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4ea03-2f14-4686-ad62-e531690a4f75 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:05.034121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7ff26-f11f-4b4d-96bc-7cbf1b9e361f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:05.034676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7ff26-f11f-4b4d-96bc-7cbf1b9e361f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.034949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7ff26-f11f-4b4d-96bc-7cbf1b9e361f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.035220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da7ff26-f11f-4b4d-96bc-7cbf1b9e361f admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:05.035872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7ff26-f11f-4b4d-96bc-7cbf1b9e361f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:45:05.036170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7ff26-f11f-4b4d-96bc-7cbf1b9e361f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:05.037816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da7ff26-f11f-4b4d-96bc-7cbf1b9e361f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:05.051846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1159/2337] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:45:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:45:05.056289 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3c303-3b3c-452e-99cd-4e92ab369960 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:05.067523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3c303-3b3c-452e-99cd-4e92ab369960 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:05.074483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5138f0-196b-4007-9e82-c581bbb0a001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:05.075406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5138f0-196b-4007-9e82-c581bbb0a001 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.075718 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5138f0-196b-4007-9e82-c581bbb0a001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.076054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5138f0-196b-4007-9e82-c581bbb0a001 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:05.076704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5138f0-196b-4007-9e82-c581bbb0a001 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:05.079498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5138f0-196b-4007-9e82-c581bbb0a001 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:05.119142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1179/2338] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:45:05 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:45:05.124272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7ff26-f11f-4b4d-96bc-7cbf1b9e361f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:05.145039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da7ff26-f11f-4b4d-96bc-7cbf1b9e361f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:05.153115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403283a-f878-4aed-9add-ff668fcde45d None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.154394 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403283a-f878-4aed-9add-ff668fcde45d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.154726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4403283a-f878-4aed-9add-ff668fcde45d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:05.180604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-4403283a-f878-4aed-9add-ff668fcde45d None None] MFA Rules not processed for user `2945d87f1c114dd0b42c4e58a8133e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:05.181970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4403283a-f878-4aed-9add-ff668fcde45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:05.221147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47311f1d-74c5-4dd1-a12c-cf758267faab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:05.221767 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311f1d-74c5-4dd1-a12c-cf758267faab admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.222042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311f1d-74c5-4dd1-a12c-cf758267faab admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.222310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47311f1d-74c5-4dd1-a12c-cf758267faab admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:05.223450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47311f1d-74c5-4dd1-a12c-cf758267faab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:05.224931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47311f1d-74c5-4dd1-a12c-cf758267faab admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:05.248321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1180/2339] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:05.251650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1160/2340] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:45:05 2022] POST /identity/v3/users => generated 286 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:45:05.257134 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403283a-f878-4aed-9add-ff668fcde45d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:05.274283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4403283a-f878-4aed-9add-ff668fcde45d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:05.279336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34cc56-7298-4b6e-b291-47c8b84148c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:05.279949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34cc56-7298-4b6e-b291-47c8b84148c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.280223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34cc56-7298-4b6e-b291-47c8b84148c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.280488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c34cc56-7298-4b6e-b291-47c8b84148c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:05.281786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34cc56-7298-4b6e-b291-47c8b84148c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:05.283367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c34cc56-7298-4b6e-b291-47c8b84148c4 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:05.303124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1181/2341] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:45:05.307117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47311f1d-74c5-4dd1-a12c-cf758267faab admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:05.319771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47311f1d-74c5-4dd1-a12c-cf758267faab admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:05.333423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a6452e-234a-44a9-a974-5c140b438877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:05.334642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6452e-234a-44a9-a974-5c140b438877 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.335417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6452e-234a-44a9-a974-5c140b438877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.336395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a6452e-234a-44a9-a974-5c140b438877 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:05.336998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6452e-234a-44a9-a974-5c140b438877 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:05.342930 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a6452e-234a-44a9-a974-5c140b438877 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:05.381003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1161/2342] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:05 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:45:05.384616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34cc56-7298-4b6e-b291-47c8b84148c4 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:05.398548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd19f44-91e9-4349-a701-f5c1b3fb7e0b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.398970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd19f44-91e9-4349-a701-f5c1b3fb7e0b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.399435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd19f44-91e9-4349-a701-f5c1b3fb7e0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:05.400861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c34cc56-7298-4b6e-b291-47c8b84148c4 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:05.411360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddadc93f-ae8c-421a-ae12-33ceb31543be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:05.412036 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadc93f-ae8c-421a-ae12-33ceb31543be admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.412307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadc93f-ae8c-421a-ae12-33ceb31543be admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.412685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddadc93f-ae8c-421a-ae12-33ceb31543be admin admin] PATH_INFO: `/v3/projects/71009736de154ddf91fca60da0c05010/users/026c9822b18548b49de763b3bb373ddd/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:05.438131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadc93f-ae8c-421a-ae12-33ceb31543be admin admin] RBAC: Authorizing `identity:create_grant(project_id=71009736de154ddf91fca60da0c05010, user_id=026c9822b18548b49de763b3bb373ddd, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:05.438480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-cdd19f44-91e9-4349-a701-f5c1b3fb7e0b None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:05.440671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cdd19f44-91e9-4349-a701-f5c1b3fb7e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:05.443240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddadc93f-ae8c-421a-ae12-33ceb31543be admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:05.477734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:05.478771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1182/2343] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:45:05 2022] PUT /identity/v3/projects/71009736de154ddf91fca60da0c05010/users/026c9822b18548b49de763b3bb373ddd/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:05.551647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205deb1f-f830-4916-8a0a-bf287dcd60b2 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.552279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205deb1f-f830-4916-8a0a-bf287dcd60b2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.552279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205deb1f-f830-4916-8a0a-bf287dcd60b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:05.559516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1162/2344] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:05.573292 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd19f44-91e9-4349-a701-f5c1b3fb7e0b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:05.575185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cdd19f44-91e9-4349-a701-f5c1b3fb7e0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:05.590002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-205deb1f-f830-4916-8a0a-bf287dcd60b2 None None] MFA Rules not processed for user `026c9822b18548b49de763b3bb373ddd`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:05.591202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-205deb1f-f830-4916-8a0a-bf287dcd60b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:05.606998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:05.673559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1183/2345] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:45:05.681471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd19f44-91e9-4349-a701-f5c1b3fb7e0b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:05.707384 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205deb1f-f830-4916-8a0a-bf287dcd60b2 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:05.708874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-205deb1f-f830-4916-8a0a-bf287dcd60b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:05.756732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9855c-d865-4bbe-9a27-03e0319525a1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:05.757465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9855c-d865-4bbe-9a27-03e0319525a1 service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.757825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9855c-d865-4bbe-9a27-03e0319525a1 service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.758160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb9855c-d865-4bbe-9a27-03e0319525a1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:05.792208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205deb1f-f830-4916-8a0a-bf287dcd60b2 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:05.801543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2c2be-cc22-42ab-b88b-770b5f234871 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42cf265450454ff9a27a29093336205e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:05.802697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2c2be-cc22-42ab-b88b-770b5f234871 service glance] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.803122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2c2be-cc22-42ab-b88b-770b5f234871 service glance] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.803535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2c2be-cc22-42ab-b88b-770b5f234871 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:05.821978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2c2be-cc22-42ab-b88b-770b5f234871 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:05.823822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2c2be-cc22-42ab-b88b-770b5f234871 service glance] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:05.829773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9855c-d865-4bbe-9a27-03e0319525a1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:05.831632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb9855c-d865-4bbe-9a27-03e0319525a1 service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:05.836479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1184/2346] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:45:05.846972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1163/2347] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:45:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 22:45:05.991801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f44ff1-3198-421a-84f5-fd9369b7f1fd None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:05.992617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f44ff1-3198-421a-84f5-fd9369b7f1fd None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:05.992952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f44ff1-3198-421a-84f5-fd9369b7f1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:06.016082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-09f44ff1-3198-421a-84f5-fd9369b7f1fd None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:06.017339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-09f44ff1-3198-421a-84f5-fd9369b7f1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:06.026696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1185/2348] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:06.044421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9855c-d865-4bbe-9a27-03e0319525a1 service swift] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:06.057645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb9855c-d865-4bbe-9a27-03e0319525a1 service swift] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:06.065733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebf86e-131b-488b-a16e-ab80e9bbe932 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:06.066497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebf86e-131b-488b-a16e-ab80e9bbe932 service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:06.066840 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebf86e-131b-488b-a16e-ab80e9bbe932 service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:06.067260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ebf86e-131b-488b-a16e-ab80e9bbe932 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:06.078221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebf86e-131b-488b-a16e-ab80e9bbe932 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:06.079991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ebf86e-131b-488b-a16e-ab80e9bbe932 service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:06.089386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1164/2349] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:45:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:45:06.424942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260c107-2285-4c79-8c2a-c52e48271838 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:06.425604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260c107-2285-4c79-8c2a-c52e48271838 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:06.426242 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e260c107-2285-4c79-8c2a-c52e48271838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:06.450869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e260c107-2285-4c79-8c2a-c52e48271838 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:06.452105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e260c107-2285-4c79-8c2a-c52e48271838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:06.462212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1186/2350] 10.222.0.39 () {58 vars in 967 bytes} [Tue Sep 6 22:45:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:06.510846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebf86e-131b-488b-a16e-ab80e9bbe932 service swift] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:06.519994 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:06.526025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ebf86e-131b-488b-a16e-ab80e9bbe932 service swift] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:06.532874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d66a4c-67f8-41f4-9b8f-4a251442a264 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:06.533621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d66a4c-67f8-41f4-9b8f-4a251442a264 service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:06.534004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d66a4c-67f8-41f4-9b8f-4a251442a264 service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:06.534390 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d66a4c-67f8-41f4-9b8f-4a251442a264 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:06.563007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d66a4c-67f8-41f4-9b8f-4a251442a264 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:06.566273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d66a4c-67f8-41f4-9b8f-4a251442a264 service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:06.576635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1165/2351] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:45:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:45:09.431358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260c107-2285-4c79-8c2a-c52e48271838 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:09.432808 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e260c107-2285-4c79-8c2a-c52e48271838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:09.538269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e260c107-2285-4c79-8c2a-c52e48271838 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:09.564592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:09.565378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:09.565722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:09.566072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] PATH_INFO: `/v3/users/1e73bc19084a4bab9727b9dda49b37c6` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:09.567325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e73bc19084a4bab9727b9dda49b37c6)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:09.569613 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:09.655133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e73bc19084a4bab9727b9dda49b37c6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:09.657094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:09.667323 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e73bc19084a4bab9727b9dda49b37c6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:09.673248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e73bc19084a4bab9727b9dda49b37c6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:09.693209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e73bc19084a4bab9727b9dda49b37c6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:09.706949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e73bc19084a4bab9727b9dda49b37c6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:09.718121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1187/2352] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:45:09 2022] DELETE /identity/v3/users/1e73bc19084a4bab9727b9dda49b37c6 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:09.720416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832354d6-3453-46b9-8346-d10afd7c7782 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:09.720986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832354d6-3453-46b9-8346-d10afd7c7782 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:09.721078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832354d6-3453-46b9-8346-d10afd7c7782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:09.743689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-832354d6-3453-46b9-8346-d10afd7c7782 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:09.744933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-832354d6-3453-46b9-8346-d10afd7c7782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:09.861224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1166/2353] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:09.871354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:09.872888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:09.963853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae1100e-2f11-443b-a544-1be1467503ba admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:09.986615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce196b8d-ff69-4262-8bc6-3d64bc34b768 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:09.987386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196b8d-ff69-4262-8bc6-3d64bc34b768 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:09.987727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196b8d-ff69-4262-8bc6-3d64bc34b768 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:09.988067 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce196b8d-ff69-4262-8bc6-3d64bc34b768 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:10.041714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce196b8d-ff69-4262-8bc6-3d64bc34b768 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:10.041714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce196b8d-ff69-4262-8bc6-3d64bc34b768 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:10.061080 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1188/2354] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:45:10.566275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832354d6-3453-46b9-8346-d10afd7c7782 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:10.575364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-832354d6-3453-46b9-8346-d10afd7c7782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:10.601997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832354d6-3453-46b9-8346-d10afd7c7782 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:10.638365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:10.639064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:10.639360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:10.639648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] PATH_INFO: `/v3/projects/ad0aee19677643cbbaeff056c8e5a2c4` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:10.642074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad0aee19677643cbbaeff056c8e5a2c4)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:10.646173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:10.743075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad0aee19677643cbbaeff056c8e5a2c4', 'request_id': 'req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:10.757362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad0aee19677643cbbaeff056c8e5a2c4', 'request_id': 'req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:10.769602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad0aee19677643cbbaeff056c8e5a2c4', 'request_id': 'req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:10.785701 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] The token cache is being invalidate because project ad0aee19677643cbbaeff056c8e5a2c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:45:10.786387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:10.787477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4ade8ae7-48f9-4d9e-9c6e-90333423f53d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:10.790618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1167/2355] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:45:10 2022] DELETE /identity/v3/projects/ad0aee19677643cbbaeff056c8e5a2c4 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:10.795870 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3afd7c-ed50-4f7b-aff3-1465cdf82fb6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:10.796454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3afd7c-ed50-4f7b-aff3-1465cdf82fb6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:10.796454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3afd7c-ed50-4f7b-aff3-1465cdf82fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:10.845219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-be3afd7c-ed50-4f7b-aff3-1465cdf82fb6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:10.846561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-be3afd7c-ed50-4f7b-aff3-1465cdf82fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:10.976773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cd097-4f1a-4001-b84e-27e8e081ad0a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:10.977120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cd097-4f1a-4001-b84e-27e8e081ad0a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:10.977395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cd097-4f1a-4001-b84e-27e8e081ad0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:10.991452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1189/2356] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:11.018465 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-550cd097-4f1a-4001-b84e-27e8e081ad0a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:11.020183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-550cd097-4f1a-4001-b84e-27e8e081ad0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:11.041396 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1168/2357] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:45:11.045917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3afd7c-ed50-4f7b-aff3-1465cdf82fb6 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:11.071081 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3afd7c-ed50-4f7b-aff3-1465cdf82fb6 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:11.103388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c20935-229c-442a-a7b9-3d00dc9da8af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:11.104268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20935-229c-442a-a7b9-3d00dc9da8af admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:11.104437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20935-229c-442a-a7b9-3d00dc9da8af admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:11.104693 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c20935-229c-442a-a7b9-3d00dc9da8af admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:11.105395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c20935-229c-442a-a7b9-3d00dc9da8af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:45:11.105688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c20935-229c-442a-a7b9-3d00dc9da8af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:11.107491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c20935-229c-442a-a7b9-3d00dc9da8af admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:11.160808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cd097-4f1a-4001-b84e-27e8e081ad0a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:11.170558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1190/2358] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:45:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:45:11.173254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cd097-4f1a-4001-b84e-27e8e081ad0a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:11.189896 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4f35b-ca77-475a-855e-236ca111a23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:11.191116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4f35b-ca77-475a-855e-236ca111a23f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:11.191568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4f35b-ca77-475a-855e-236ca111a23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:11.192002 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4f35b-ca77-475a-855e-236ca111a23f admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:11.196946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4f35b-ca77-475a-855e-236ca111a23f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:11.199197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4f35b-ca77-475a-855e-236ca111a23f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:11.288600 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1169/2359] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:45:11 2022] POST /identity/v3/projects => generated 377 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:45:11.302643 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c20935-229c-442a-a7b9-3d00dc9da8af admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:11.315830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c20935-229c-442a-a7b9-3d00dc9da8af admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:11.429459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:INSERT=1,keystone:DELETE=3,keystone:SELECT=96 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:11.440424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c7528-5138-4260-94dc-0d8afcecfe88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:11.442916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c7528-5138-4260-94dc-0d8afcecfe88 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:11.444450 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c7528-5138-4260-94dc-0d8afcecfe88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:11.445245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34c7528-5138-4260-94dc-0d8afcecfe88 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:11.446325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c7528-5138-4260-94dc-0d8afcecfe88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:11.451053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34c7528-5138-4260-94dc-0d8afcecfe88 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:11.505947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1191/2360] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:45:11 2022] POST /identity/v3/users => generated 289 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:45:11.511272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4f35b-ca77-475a-855e-236ca111a23f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:11.532715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4f35b-ca77-475a-855e-236ca111a23f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:11.553202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05155ffe-9717-40c6-9188-ae15e063b924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:11.553202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05155ffe-9717-40c6-9188-ae15e063b924 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:11.553202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05155ffe-9717-40c6-9188-ae15e063b924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:11.553202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05155ffe-9717-40c6-9188-ae15e063b924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:11.553674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05155ffe-9717-40c6-9188-ae15e063b924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:11.554609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05155ffe-9717-40c6-9188-ae15e063b924 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:11.576613 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1170/2361] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:45:11.580688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c7528-5138-4260-94dc-0d8afcecfe88 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:11.590918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34c7528-5138-4260-94dc-0d8afcecfe88 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:11.607660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e47994-2431-41e2-90c8-d4554f36b42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:11.608450 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e47994-2431-41e2-90c8-d4554f36b42b admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:11.608621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e47994-2431-41e2-90c8-d4554f36b42b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:11.609840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e47994-2431-41e2-90c8-d4554f36b42b admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:11.611085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e47994-2431-41e2-90c8-d4554f36b42b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:11.612872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e47994-2431-41e2-90c8-d4554f36b42b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:11.635702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1192/2362] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:45:11.637501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05155ffe-9717-40c6-9188-ae15e063b924 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:11.657709 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05155ffe-9717-40c6-9188-ae15e063b924 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:11.666960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa3804-f76d-48bf-a1b6-5c09c82394cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:11.667710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa3804-f76d-48bf-a1b6-5c09c82394cc admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:11.667991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa3804-f76d-48bf-a1b6-5c09c82394cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:11.668321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa3804-f76d-48bf-a1b6-5c09c82394cc admin admin] PATH_INFO: `/v3/projects/cc47231427ec4337833eb5d733b211bc/users/29018ec15cc147d3919d9047569a4c68/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:11.716823 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa3804-f76d-48bf-a1b6-5c09c82394cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc47231427ec4337833eb5d733b211bc, user_id=29018ec15cc147d3919d9047569a4c68, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:11.720292 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa3804-f76d-48bf-a1b6-5c09c82394cc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:11.746829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1171/2363] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:45:11 2022] PUT /identity/v3/projects/cc47231427ec4337833eb5d733b211bc/users/29018ec15cc147d3919d9047569a4c68/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:11.756202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7ec61-877f-4c6f-b02f-d154c6cdcfa9 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:11.757006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7ec61-877f-4c6f-b02f-d154c6cdcfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:11.757390 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7ec61-877f-4c6f-b02f-d154c6cdcfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:11.794135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-f7e7ec61-877f-4c6f-b02f-d154c6cdcfa9 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:11.795577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f7e7ec61-877f-4c6f-b02f-d154c6cdcfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:11.898150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1193/2364] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:11.905155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa3804-f76d-48bf-a1b6-5c09c82394cc admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:11.909775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4bfa3804-f76d-48bf-a1b6-5c09c82394cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:11.930036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:11.991496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa3804-f76d-48bf-a1b6-5c09c82394cc admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:11.997406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047e2fa-fe43-473b-9b84-45be174f4bd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:11.998091 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047e2fa-fe43-473b-9b84-45be174f4bd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:11.998370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047e2fa-fe43-473b-9b84-45be174f4bd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:11.998666 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047e2fa-fe43-473b-9b84-45be174f4bd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:12.009786 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047e2fa-fe43-473b-9b84-45be174f4bd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:12.011241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047e2fa-fe43-473b-9b84-45be174f4bd9 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:12.033191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1172/2365] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:45:20.720615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ce9eb-507e-4ef1-9d0d-9165be594820 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:20.721185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ce9eb-507e-4ef1-9d0d-9165be594820 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:20.721282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ce9eb-507e-4ef1-9d0d-9165be594820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:20.767145 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-8b9ce9eb-507e-4ef1-9d0d-9165be594820 None None] MFA Rules not processed for user `29018ec15cc147d3919d9047569a4c68`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:20.768413 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8b9ce9eb-507e-4ef1-9d0d-9165be594820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:20.863584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1194/2366] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:20 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:20.865373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047e2fa-fe43-473b-9b84-45be174f4bd9 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:20.876550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047e2fa-fe43-473b-9b84-45be174f4bd9 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:20.882778 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb976-6ac6-48e3-871a-ad20e9933360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:20.883382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb976-6ac6-48e3-871a-ad20e9933360 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:20.883661 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb976-6ac6-48e3-871a-ad20e9933360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:20.883936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5fb976-6ac6-48e3-871a-ad20e9933360 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:20.884566 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb976-6ac6-48e3-871a-ad20e9933360 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:20.886534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5fb976-6ac6-48e3-871a-ad20e9933360 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:20.914503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1173/2367] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:45:20 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:45:20.918588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ce9eb-507e-4ef1-9d0d-9165be594820 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:20.931505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ce9eb-507e-4ef1-9d0d-9165be594820 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:20.992544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082b751-9638-4029-80ab-c972021af84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:20.993183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082b751-9638-4029-80ab-c972021af84a admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:20.993456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082b751-9638-4029-80ab-c972021af84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:20.993720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c082b751-9638-4029-80ab-c972021af84a admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:20.994386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082b751-9638-4029-80ab-c972021af84a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:20.996271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c082b751-9638-4029-80ab-c972021af84a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:21.007765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:21.022290 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1195/2368] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:45:20 2022] POST /identity/v3/users => generated 295 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:45:21.025967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb976-6ac6-48e3-871a-ad20e9933360 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:21.039183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5fb976-6ac6-48e3-871a-ad20e9933360 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:21.044068 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de767a83-af52-46c3-858a-f7d7454e5c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:21.044811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de767a83-af52-46c3-858a-f7d7454e5c08 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:21.045047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de767a83-af52-46c3-858a-f7d7454e5c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:21.045725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de767a83-af52-46c3-858a-f7d7454e5c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:21.047074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de767a83-af52-46c3-858a-f7d7454e5c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:21.049129 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de767a83-af52-46c3-858a-f7d7454e5c08 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:21.061706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1174/2369] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:45:21.065927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082b751-9638-4029-80ab-c972021af84a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:21.076354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c082b751-9638-4029-80ab-c972021af84a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:21.086196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df93a1-446a-488e-a9ab-59d9aa5fb559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:21.087032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df93a1-446a-488e-a9ab-59d9aa5fb559 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:21.087315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df93a1-446a-488e-a9ab-59d9aa5fb559 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:21.087610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5df93a1-446a-488e-a9ab-59d9aa5fb559 admin admin] PATH_INFO: `/v3/projects/91d8ca3743ba41018cc97716004fcce9/users/8d0a2c6199534477b7aac1154cc53d4c/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:21.106056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df93a1-446a-488e-a9ab-59d9aa5fb559 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d8ca3743ba41018cc97716004fcce9, user_id=8d0a2c6199534477b7aac1154cc53d4c, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:21.108722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5df93a1-446a-488e-a9ab-59d9aa5fb559 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:21.129283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de767a83-af52-46c3-858a-f7d7454e5c08 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:21.129792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1196/2370] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:45:21 2022] PUT /identity/v3/projects/91d8ca3743ba41018cc97716004fcce9/users/8d0a2c6199534477b7aac1154cc53d4c/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:45:21.141660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de767a83-af52-46c3-858a-f7d7454e5c08 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:21.210267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc93eaf1-39ab-478e-91bb-b10ddd2d0416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:21.210982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93eaf1-39ab-478e-91bb-b10ddd2d0416 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:21.211334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93eaf1-39ab-478e-91bb-b10ddd2d0416 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:21.211670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc93eaf1-39ab-478e-91bb-b10ddd2d0416 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:21.212382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc93eaf1-39ab-478e-91bb-b10ddd2d0416 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:21.214647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc93eaf1-39ab-478e-91bb-b10ddd2d0416 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:21.230054 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1175/2371] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:21 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:45:21.231273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df93a1-446a-488e-a9ab-59d9aa5fb559 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:21.243877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5df93a1-446a-488e-a9ab-59d9aa5fb559 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:21.248700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c3a40-c1d6-4324-be83-7239b5ed237d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:21.249388 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c3a40-c1d6-4324-be83-7239b5ed237d admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:21.250284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c3a40-c1d6-4324-be83-7239b5ed237d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:21.250561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6c3a40-c1d6-4324-be83-7239b5ed237d admin admin] PATH_INFO: `/v3/projects/91d8ca3743ba41018cc97716004fcce9/users/8d0a2c6199534477b7aac1154cc53d4c/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:21.252498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c3a40-c1d6-4324-be83-7239b5ed237d admin admin] RBAC: Authorizing `identity:create_grant(project_id=91d8ca3743ba41018cc97716004fcce9, user_id=8d0a2c6199534477b7aac1154cc53d4c, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:21.255209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6c3a40-c1d6-4324-be83-7239b5ed237d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:21.265798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1197/2372] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:45:21 2022] PUT /identity/v3/projects/91d8ca3743ba41018cc97716004fcce9/users/8d0a2c6199534477b7aac1154cc53d4c/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:27.489505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0b251-7108-4e04-aee2-9aadb1674249 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:27.489994 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0b251-7108-4e04-aee2-9aadb1674249 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:27.490195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a0b251-7108-4e04-aee2-9aadb1674249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:27.530102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-58a0b251-7108-4e04-aee2-9aadb1674249 None None] MFA Rules not processed for user `8d0a2c6199534477b7aac1154cc53d4c`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:27.532271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-58a0b251-7108-4e04-aee2-9aadb1674249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:27.620741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1176/2373] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:27 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:27.634742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c3a40-c1d6-4324-be83-7239b5ed237d admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:27.636363 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-bd6c3a40-c1d6-4324-be83-7239b5ed237d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:27.719544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6c3a40-c1d6-4324-be83-7239b5ed237d admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:27.727310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3c96d-f987-443a-9157-ea990c0df397 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:27.727962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3c96d-f987-443a-9157-ea990c0df397 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:27.728238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3c96d-f987-443a-9157-ea990c0df397 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:27.728655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3c96d-f987-443a-9157-ea990c0df397 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:27.791519 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3c96d-f987-443a-9157-ea990c0df397 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:27.793416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3c96d-f987-443a-9157-ea990c0df397 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:27.803768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1198/2374] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:27 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:45:31.058355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc92202-fa7f-499e-b2d9-4ff7133342ee None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:31.058952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc92202-fa7f-499e-b2d9-4ff7133342ee None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:31.058952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc92202-fa7f-499e-b2d9-4ff7133342ee None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:31.060502 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1177/2375] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:45:31.071186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8afc3-be20-4de4-84e7-36b4e2af50a3 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:31.072236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8afc3-be20-4de4-84e7-36b4e2af50a3 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:31.072675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac8afc3-be20-4de4-84e7-36b4e2af50a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:31.097828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7ac8afc3-be20-4de4-84e7-36b4e2af50a3 None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:31.099647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7ac8afc3-be20-4de4-84e7-36b4e2af50a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:31.114154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1199/2376] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:31.132124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc92202-fa7f-499e-b2d9-4ff7133342ee None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:31.134470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1bc92202-fa7f-499e-b2d9-4ff7133342ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:31.147249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:31.240378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc92202-fa7f-499e-b2d9-4ff7133342ee None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:31.244368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa7262-de36-46b1-b96e-fe7e77762bfc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42cf265450454ff9a27a29093336205e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:31.245245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa7262-de36-46b1-b96e-fe7e77762bfc service glance] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:31.245851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa7262-de36-46b1-b96e-fe7e77762bfc service glance] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:31.246266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa7262-de36-46b1-b96e-fe7e77762bfc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:31.259286 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa7262-de36-46b1-b96e-fe7e77762bfc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:31.261008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa7262-de36-46b1-b96e-fe7e77762bfc service glance] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:31.279985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1178/2377] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:45:31.382244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f7387-0b1b-447a-bc04-60ee9fdfd248 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:31.382530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f7387-0b1b-447a-bc04-60ee9fdfd248 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:31.382805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458f7387-0b1b-447a-bc04-60ee9fdfd248 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:31.384324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1200/2378] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:45:31.389467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e20c2-981f-4e20-aa0e-829625f4efaa None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:31.389748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e20c2-981f-4e20-aa0e-829625f4efaa None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:31.390596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7e20c2-981f-4e20-aa0e-829625f4efaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:31.413441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-da7e20c2-981f-4e20-aa0e-829625f4efaa None None] MFA Rules not processed for user `2658435f74a947f39e207465ebff22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:31.414700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-da7e20c2-981f-4e20-aa0e-829625f4efaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:31.479274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1179/2379] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:45:31.491247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f7387-0b1b-447a-bc04-60ee9fdfd248 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:31.504017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458f7387-0b1b-447a-bc04-60ee9fdfd248 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:31.508406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bedc32-4583-4799-ac2e-1001d14d2cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:31.509130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bedc32-4583-4799-ac2e-1001d14d2cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:31.509594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bedc32-4583-4799-ac2e-1001d14d2cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:31.509865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bedc32-4583-4799-ac2e-1001d14d2cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:31.518804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bedc32-4583-4799-ac2e-1001d14d2cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:31.520251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bedc32-4583-4799-ac2e-1001d14d2cf3 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:31.529068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1201/2380] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:31 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:45:31.653544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfc9fc-86e0-43a2-a9db-b4a5d0a1b4c4 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:31.653833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfc9fc-86e0-43a2-a9db-b4a5d0a1b4c4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:31.654113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cfc9fc-86e0-43a2-a9db-b4a5d0a1b4c4 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:31.655468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1180/2381] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:45:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:45:31.661089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebc39d-4fe1-4b13-ae76-ecfbd039c863 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:31.661372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebc39d-4fe1-4b13-ae76-ecfbd039c863 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:31.661639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebc39d-4fe1-4b13-ae76-ecfbd039c863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:31.682016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-77ebc39d-4fe1-4b13-ae76-ecfbd039c863 None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:31.683251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-77ebc39d-4fe1-4b13-ae76-ecfbd039c863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:31.693673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1202/2382] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:45:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:31.709967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfc9fc-86e0-43a2-a9db-b4a5d0a1b4c4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:31.711865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d1cfc9fc-86e0-43a2-a9db-b4a5d0a1b4c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:31.725796 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1cfc9fc-86e0-43a2-a9db-b4a5d0a1b4c4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:31.729389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579b262c-a336-4bdc-bffb-72620010d3c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:31.730006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b262c-a336-4bdc-bffb-72620010d3c7 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:31.730277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b262c-a336-4bdc-bffb-72620010d3c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:31.730541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579b262c-a336-4bdc-bffb-72620010d3c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:31.742530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b262c-a336-4bdc-bffb-72620010d3c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:31.743878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579b262c-a336-4bdc-bffb-72620010d3c7 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:31.760105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1181/2383] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:45:37.340472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebc39d-4fe1-4b13-ae76-ecfbd039c863 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:37.341948 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-77ebc39d-4fe1-4b13-ae76-ecfbd039c863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:37.377039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:37.416465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebc39d-4fe1-4b13-ae76-ecfbd039c863 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:37.422950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:37.423088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:37.423843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:37.424160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] PATH_INFO: `/v3/users/026c9822b18548b49de763b3bb373ddd` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:37.425788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] RBAC: Authorizing `identity:delete_user(user_id=026c9822b18548b49de763b3bb373ddd)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:37.431751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:37.488347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '026c9822b18548b49de763b3bb373ddd'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:37.495692 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '026c9822b18548b49de763b3bb373ddd'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:37.496510 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '026c9822b18548b49de763b3bb373ddd'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:37.513753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '026c9822b18548b49de763b3bb373ddd'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:37.522337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '026c9822b18548b49de763b3bb373ddd'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:37.532921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1203/2384] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:45:37 2022] DELETE /identity/v3/users/026c9822b18548b49de763b3bb373ddd => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:37.537911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b470a3-9041-4510-8ea9-0636d49cb17b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:37.538468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b470a3-9041-4510-8ea9-0636d49cb17b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:37.538688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b470a3-9041-4510-8ea9-0636d49cb17b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:37.593673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-38b470a3-9041-4510-8ea9-0636d49cb17b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:37.595922 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-38b470a3-9041-4510-8ea9-0636d49cb17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:37.663334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1182/2385] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:37.673190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:37.674529 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:37.732461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c9752a-5244-4471-94ec-05c22ef47fee admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:37.736605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5baa69-b722-4c3a-8ee5-a6b97edba808 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:37.737253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5baa69-b722-4c3a-8ee5-a6b97edba808 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:37.737532 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5baa69-b722-4c3a-8ee5-a6b97edba808 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:37.737799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5baa69-b722-4c3a-8ee5-a6b97edba808 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:37.747273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5baa69-b722-4c3a-8ee5-a6b97edba808 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:37.748610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5baa69-b722-4c3a-8ee5-a6b97edba808 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:37.757976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1204/2386] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:45:38.047187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b470a3-9041-4510-8ea9-0636d49cb17b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:38.049299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-38b470a3-9041-4510-8ea9-0636d49cb17b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:38.072121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b470a3-9041-4510-8ea9-0636d49cb17b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:38.086978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:38.087627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:38.087905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:38.088175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] PATH_INFO: `/v3/projects/71009736de154ddf91fca60da0c05010` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:38.089323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71009736de154ddf91fca60da0c05010)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:38.091214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:38.137478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:38.149576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71009736de154ddf91fca60da0c05010', 'request_id': 'req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:38.156251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71009736de154ddf91fca60da0c05010', 'request_id': 'req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:38.163763 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71009736de154ddf91fca60da0c05010', 'request_id': 'req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:38.175034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] The token cache is being invalidate because project 71009736de154ddf91fca60da0c05010 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:45:38.175320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:38.175950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-fdb0d87a-7f73-4c1e-9d0c-74ea2f7de966 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:45:38.177635 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1183/2387] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:45:38 2022] DELETE /identity/v3/projects/71009736de154ddf91fca60da0c05010 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:38.182728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab9253-a2a1-4778-a23b-011393c9f7ea None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:38.183012 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab9253-a2a1-4778-a23b-011393c9f7ea None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:38.183315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab9253-a2a1-4778-a23b-011393c9f7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:38.204457 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-00ab9253-a2a1-4778-a23b-011393c9f7ea None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:38.205715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-00ab9253-a2a1-4778-a23b-011393c9f7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:38.257738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:38.279935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1205/2388] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:38.289094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8403-d9d8-44a1-ab14-076cf44472cb None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:38.289243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8403-d9d8-44a1-ab14-076cf44472cb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:38.289515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e8403-d9d8-44a1-ab14-076cf44472cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:38.313020 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-041e8403-d9d8-44a1-ab14-076cf44472cb None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:38.314378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-041e8403-d9d8-44a1-ab14-076cf44472cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:38.325302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1184/2389] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:38.329959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab9253-a2a1-4778-a23b-011393c9f7ea None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:38.340801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab9253-a2a1-4778-a23b-011393c9f7ea None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:38.345696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749882a-e40e-454f-9147-2a22696d042a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:38.346414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749882a-e40e-454f-9147-2a22696d042a admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:38.346766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749882a-e40e-454f-9147-2a22696d042a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:38.347043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6749882a-e40e-454f-9147-2a22696d042a admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:38.347695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749882a-e40e-454f-9147-2a22696d042a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:45:38.347993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749882a-e40e-454f-9147-2a22696d042a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:38.349613 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6749882a-e40e-454f-9147-2a22696d042a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:38.361806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1206/2390] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:45:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:45:38.366140 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8403-d9d8-44a1-ab14-076cf44472cb None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:38.375337 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e8403-d9d8-44a1-ab14-076cf44472cb None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:38.379995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d3890-9ce3-4097-bde1-aa4b28d37935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:38.380592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3890-9ce3-4097-bde1-aa4b28d37935 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:38.380940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3890-9ce3-4097-bde1-aa4b28d37935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:38.381163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8d3890-9ce3-4097-bde1-aa4b28d37935 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:38.381875 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d3890-9ce3-4097-bde1-aa4b28d37935 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:38.383790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8d3890-9ce3-4097-bde1-aa4b28d37935 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:38.414055 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1185/2391] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:45:38 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:45:38.418220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749882a-e40e-454f-9147-2a22696d042a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:38.429010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6749882a-e40e-454f-9147-2a22696d042a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:38.496035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e5c0f-0553-43f4-91c2-de18d174c9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:38.496943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e5c0f-0553-43f4-91c2-de18d174c9bf admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:38.497219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e5c0f-0553-43f4-91c2-de18d174c9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:38.497490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4e5c0f-0553-43f4-91c2-de18d174c9bf admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:38.498187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e5c0f-0553-43f4-91c2-de18d174c9bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:38.500686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4e5c0f-0553-43f4-91c2-de18d174c9bf admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:38.540673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1207/2392] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:45:38 2022] POST /identity/v3/users => generated 286 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:45:38.541878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d3890-9ce3-4097-bde1-aa4b28d37935 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:38.556153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8d3890-9ce3-4097-bde1-aa4b28d37935 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:38.561555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa9f19-52ee-46c9-bfd3-7c4c2e5da782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:38.562309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa9f19-52ee-46c9-bfd3-7c4c2e5da782 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:38.562939 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa9f19-52ee-46c9-bfd3-7c4c2e5da782 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:38.563210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa9f19-52ee-46c9-bfd3-7c4c2e5da782 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:38.563854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa9f19-52ee-46c9-bfd3-7c4c2e5da782 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:38.566221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa9f19-52ee-46c9-bfd3-7c4c2e5da782 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:38.581030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1186/2393] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:45:38.581882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e5c0f-0553-43f4-91c2-de18d174c9bf admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:38.596150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4e5c0f-0553-43f4-91c2-de18d174c9bf admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:38.601502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc2456-75c8-4e8c-acad-7cb133d16447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:38.602237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc2456-75c8-4e8c-acad-7cb133d16447 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:38.602615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc2456-75c8-4e8c-acad-7cb133d16447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:38.603018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc2456-75c8-4e8c-acad-7cb133d16447 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:38.606083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc2456-75c8-4e8c-acad-7cb133d16447 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:38.608193 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc2456-75c8-4e8c-acad-7cb133d16447 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:38.621904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1208/2394] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:45:38.626537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa9f19-52ee-46c9-bfd3-7c4c2e5da782 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:38.637790 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa9f19-52ee-46c9-bfd3-7c4c2e5da782 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:38.645716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce230d60-e68f-4321-8de8-e857a1940b1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:38.646992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce230d60-e68f-4321-8de8-e857a1940b1d admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:38.647437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce230d60-e68f-4321-8de8-e857a1940b1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:38.647871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce230d60-e68f-4321-8de8-e857a1940b1d admin admin] PATH_INFO: `/v3/projects/410f8fc9cc284ba78edf9f4db25f9d31/users/bcede7fa39dd47f7ba80c5add2c0f750/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:38.669304 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce230d60-e68f-4321-8de8-e857a1940b1d admin admin] RBAC: Authorizing `identity:create_grant(project_id=410f8fc9cc284ba78edf9f4db25f9d31, user_id=bcede7fa39dd47f7ba80c5add2c0f750, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:38.672308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce230d60-e68f-4321-8de8-e857a1940b1d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:38.694701 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1187/2395] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:45:38 2022] PUT /identity/v3/projects/410f8fc9cc284ba78edf9f4db25f9d31/users/bcede7fa39dd47f7ba80c5add2c0f750/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:38.699545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec33ec9-f392-4725-811d-48d32bfab110 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:38.699827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec33ec9-f392-4725-811d-48d32bfab110 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:38.700803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec33ec9-f392-4725-811d-48d32bfab110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:38.720753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-dec33ec9-f392-4725-811d-48d32bfab110 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:38.722112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-dec33ec9-f392-4725-811d-48d32bfab110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:38.791859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1209/2396] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:38.799230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce230d60-e68f-4321-8de8-e857a1940b1d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:38.800992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ce230d60-e68f-4321-8de8-e857a1940b1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:38.866424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce230d60-e68f-4321-8de8-e857a1940b1d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:38.870209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9147f-3267-4901-9974-dff8bc355e79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:38.870830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9147f-3267-4901-9974-dff8bc355e79 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:38.871134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9147f-3267-4901-9974-dff8bc355e79 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:38.871419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9147f-3267-4901-9974-dff8bc355e79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:38.883058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9147f-3267-4901-9974-dff8bc355e79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:38.885338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9147f-3267-4901-9974-dff8bc355e79 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:38.899593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1188/2397] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:45:44.363725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788997e-616a-4939-a6e0-88f6321a83a0 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:44.364197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788997e-616a-4939-a6e0-88f6321a83a0 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:44.364292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788997e-616a-4939-a6e0-88f6321a83a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:44.400560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-1788997e-616a-4939-a6e0-88f6321a83a0 None None] MFA Rules not processed for user `bcede7fa39dd47f7ba80c5add2c0f750`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:44.402156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1788997e-616a-4939-a6e0-88f6321a83a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:44.474043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:44.505201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1210/2398] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:44.509538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9147f-3267-4901-9974-dff8bc355e79 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:44.521265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9147f-3267-4901-9974-dff8bc355e79 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:44.529472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3af91d-680d-42d1-b636-572bcd5fbf08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:44.530089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3af91d-680d-42d1-b636-572bcd5fbf08 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:44.530352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3af91d-680d-42d1-b636-572bcd5fbf08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:44.530652 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3af91d-680d-42d1-b636-572bcd5fbf08 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:44.531426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3af91d-680d-42d1-b636-572bcd5fbf08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:44.533335 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3af91d-680d-42d1-b636-572bcd5fbf08 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:44.618238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1189/2399] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:45:44 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:45:44.622107 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788997e-616a-4939-a6e0-88f6321a83a0 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:44.641237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788997e-616a-4939-a6e0-88f6321a83a0 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:44.724010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbc688-d9fc-484c-957e-942013e5afb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:44.724610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbc688-d9fc-484c-957e-942013e5afb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:44.724922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbc688-d9fc-484c-957e-942013e5afb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:44.725194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbbc688-d9fc-484c-957e-942013e5afb0 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:44.725817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbc688-d9fc-484c-957e-942013e5afb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:44.727766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bbbc688-d9fc-484c-957e-942013e5afb0 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:44.748716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1211/2400] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:45:44 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:45:44.753626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3af91d-680d-42d1-b636-572bcd5fbf08 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:44.764384 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3af91d-680d-42d1-b636-572bcd5fbf08 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:44.770240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a43b07-aeed-4351-86ec-2b1f8b9c65aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:44.770816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a43b07-aeed-4351-86ec-2b1f8b9c65aa admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:44.771095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a43b07-aeed-4351-86ec-2b1f8b9c65aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:44.771627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a43b07-aeed-4351-86ec-2b1f8b9c65aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:44.772264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a43b07-aeed-4351-86ec-2b1f8b9c65aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:44.773893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a43b07-aeed-4351-86ec-2b1f8b9c65aa admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:44.785076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1190/2401] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:45:44.789254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbc688-d9fc-484c-957e-942013e5afb0 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:44.799136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bbbc688-d9fc-484c-957e-942013e5afb0 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:44.803863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be0aad-aa52-407c-bbc4-e2b9712da224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:44.804502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be0aad-aa52-407c-bbc4-e2b9712da224 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:44.804813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be0aad-aa52-407c-bbc4-e2b9712da224 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:44.805091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7be0aad-aa52-407c-bbc4-e2b9712da224 admin admin] PATH_INFO: `/v3/projects/c6190732d5f743e6ab4e220ea6aad81d/users/365b65fe391445a7a4693d9d445203e9/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:44.820695 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be0aad-aa52-407c-bbc4-e2b9712da224 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6190732d5f743e6ab4e220ea6aad81d, user_id=365b65fe391445a7a4693d9d445203e9, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:44.823354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7be0aad-aa52-407c-bbc4-e2b9712da224 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:44.841564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1212/2402] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:45:44 2022] PUT /identity/v3/projects/c6190732d5f743e6ab4e220ea6aad81d/users/365b65fe391445a7a4693d9d445203e9/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:44.846383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a43b07-aeed-4351-86ec-2b1f8b9c65aa admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:44.856793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a43b07-aeed-4351-86ec-2b1f8b9c65aa admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:44.886734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:44.916165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09d294-fa1d-41fc-9af0-48be84ef6843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:44.916840 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09d294-fa1d-41fc-9af0-48be84ef6843 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:44.917120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09d294-fa1d-41fc-9af0-48be84ef6843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:44.917386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09d294-fa1d-41fc-9af0-48be84ef6843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:44.918023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09d294-fa1d-41fc-9af0-48be84ef6843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:44.919597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09d294-fa1d-41fc-9af0-48be84ef6843 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:44.929815 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1191/2403] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:45:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:45:44.934091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be0aad-aa52-407c-bbc4-e2b9712da224 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:44.945071 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7be0aad-aa52-407c-bbc4-e2b9712da224 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:44.949733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72895ef5-c593-40a0-8d9c-feca4a3c6e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:44.950369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72895ef5-c593-40a0-8d9c-feca4a3c6e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:44.950640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72895ef5-c593-40a0-8d9c-feca4a3c6e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:44.950903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72895ef5-c593-40a0-8d9c-feca4a3c6e18 admin admin] PATH_INFO: `/v3/projects/c6190732d5f743e6ab4e220ea6aad81d/users/365b65fe391445a7a4693d9d445203e9/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:44.953095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72895ef5-c593-40a0-8d9c-feca4a3c6e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6190732d5f743e6ab4e220ea6aad81d, user_id=365b65fe391445a7a4693d9d445203e9, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:44.955512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72895ef5-c593-40a0-8d9c-feca4a3c6e18 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:44.966098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1213/2404] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:45:44 2022] PUT /identity/v3/projects/c6190732d5f743e6ab4e220ea6aad81d/users/365b65fe391445a7a4693d9d445203e9/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:45:47.833216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09d294-fa1d-41fc-9af0-48be84ef6843 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:47.835173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cb09d294-fa1d-41fc-9af0-48be84ef6843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:48.003945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09d294-fa1d-41fc-9af0-48be84ef6843 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:48.114233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9716f-5121-4780-874e-2b40fd293bf7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:48.114866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9716f-5121-4780-874e-2b40fd293bf7 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:48.115213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9716f-5121-4780-874e-2b40fd293bf7 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:48.115425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9716f-5121-4780-874e-2b40fd293bf7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:48.121464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a0e9716f-5121-4780-874e-2b40fd293bf7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:48.165024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9716f-5121-4780-874e-2b40fd293bf7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:48.167948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e9716f-5121-4780-874e-2b40fd293bf7 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:48.198840 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1192/2405] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:45:50.406290 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f742415-3cab-45bf-a2e2-de79a4c1460c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:50.406766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f742415-3cab-45bf-a2e2-de79a4c1460c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:50.407114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f742415-3cab-45bf-a2e2-de79a4c1460c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:50.447702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2f742415-3cab-45bf-a2e2-de79a4c1460c None None] MFA Rules not processed for user `29018ec15cc147d3919d9047569a4c68`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:50.450727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2f742415-3cab-45bf-a2e2-de79a4c1460c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:50.557409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1214/2406] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:50.613230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61304d23-ba82-47d4-a1c9-1e6affca5e99 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:50.613808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61304d23-ba82-47d4-a1c9-1e6affca5e99 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:50.613808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61304d23-ba82-47d4-a1c9-1e6affca5e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:50.656080 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-61304d23-ba82-47d4-a1c9-1e6affca5e99 None None] MFA Rules not processed for user `365b65fe391445a7a4693d9d445203e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:50.657410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-61304d23-ba82-47d4-a1c9-1e6affca5e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:50.748926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:50.784327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1193/2407] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:50 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:50.800230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f742415-3cab-45bf-a2e2-de79a4c1460c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:50.801858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2f742415-3cab-45bf-a2e2-de79a4c1460c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:50.868888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:45:50.907495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f742415-3cab-45bf-a2e2-de79a4c1460c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:50.913253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67cd2f-d356-48be-80a5-2449084dbf56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:50.914046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67cd2f-d356-48be-80a5-2449084dbf56 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:50.914333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67cd2f-d356-48be-80a5-2449084dbf56 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:50.914608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a67cd2f-d356-48be-80a5-2449084dbf56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:50.990179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67cd2f-d356-48be-80a5-2449084dbf56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:50.991792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a67cd2f-d356-48be-80a5-2449084dbf56 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:51.007250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1215/2408] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:45:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:45:52.080836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e8c26-7f80-48ab-903f-2284747b3800 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:52.081344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e8c26-7f80-48ab-903f-2284747b3800 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:52.082313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e8c26-7f80-48ab-903f-2284747b3800 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:52.111099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-264e8c26-7f80-48ab-903f-2284747b3800 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:52.112341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-264e8c26-7f80-48ab-903f-2284747b3800 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:52.231753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1194/2409] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:45:52 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:45:52.248240 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67cd2f-d356-48be-80a5-2449084dbf56 service nova] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:45:52.251227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1a67cd2f-d356-48be-80a5-2449084dbf56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:52.336674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a67cd2f-d356-48be-80a5-2449084dbf56 service nova] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:45:52.347533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab027049-e549-46b3-946f-caaf8b11d179 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:45:52.348538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027049-e549-46b3-946f-caaf8b11d179 service swift] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:52.348852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027049-e549-46b3-946f-caaf8b11d179 service swift] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:52.349133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab027049-e549-46b3-946f-caaf8b11d179 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:52.361113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab027049-e549-46b3-946f-caaf8b11d179 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:45:52.363274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab027049-e549-46b3-946f-caaf8b11d179 service swift] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:45:52.377054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1216/2410] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:45:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:45:52.845975 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f54f1-bfe4-400b-855d-4ade50df2818 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:45:52.846094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f54f1-bfe4-400b-855d-4ade50df2818 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:45:52.846356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f54f1-bfe4-400b-855d-4ade50df2818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:45:52.866624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e34f54f1-bfe4-400b-855d-4ade50df2818 None None] MFA Rules not processed for user `bcede7fa39dd47f7ba80c5add2c0f750`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:45:52.867996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e34f54f1-bfe4-400b-855d-4ade50df2818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:45:52.875270 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1195/2411] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:45:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:46:00.835713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961242f-85d1-4ad3-b8fe-fbbfbe5576d1 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:46:00.836277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961242f-85d1-4ad3-b8fe-fbbfbe5576d1 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:46:00.836277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8961242f-85d1-4ad3-b8fe-fbbfbe5576d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:46:00.860329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-8961242f-85d1-4ad3-b8fe-fbbfbe5576d1 None None] MFA Rules not processed for user `81d8036279ef4384bdbfa8c37e46a1d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:46:00.861716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8961242f-85d1-4ad3-b8fe-fbbfbe5576d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:46:00.956013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1217/2412] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:46:00 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:46:02.861508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=35 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:46:10.948312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=94 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:46:18.129785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1de1-2573-4c2e-90bd-eeb358fa02a7 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:46:18.130342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1de1-2573-4c2e-90bd-eeb358fa02a7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:46:18.130342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddc1de1-2573-4c2e-90bd-eeb358fa02a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:46:18.168396 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-eddc1de1-2573-4c2e-90bd-eeb358fa02a7 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:46:18.169825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-eddc1de1-2573-4c2e-90bd-eeb358fa02a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:46:18.178665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1196/2413] 10.222.0.39 () {58 vars in 967 bytes} [Tue Sep 6 22:46:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:46:18.204307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961242f-85d1-4ad3-b8fe-fbbfbe5576d1 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:46:18.216818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8961242f-85d1-4ad3-b8fe-fbbfbe5576d1 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:46:18.224042 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0947321-fd6e-4a26-9ebb-24677d920cad service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:46:18.225474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0947321-fd6e-4a26-9ebb-24677d920cad service swift] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:46:18.225748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0947321-fd6e-4a26-9ebb-24677d920cad service swift] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:46:18.226020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0947321-fd6e-4a26-9ebb-24677d920cad service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:46:18.236523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0947321-fd6e-4a26-9ebb-24677d920cad service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:46:18.238188 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0947321-fd6e-4a26-9ebb-24677d920cad service swift] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:46:18.246174 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1218/2414] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:46:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:46:24.246409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1de1-2573-4c2e-90bd-eeb358fa02a7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:46:24.247835 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-eddc1de1-2573-4c2e-90bd-eeb358fa02a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:46:24.304984 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddc1de1-2573-4c2e-90bd-eeb358fa02a7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:46:24.308471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d191d5d7-00a5-4d78-8ea2-fe69d36c1d5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:46:24.309092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191d5d7-00a5-4d78-8ea2-fe69d36c1d5c service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:46:24.309361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191d5d7-00a5-4d78-8ea2-fe69d36c1d5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:46:24.309623 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d191d5d7-00a5-4d78-8ea2-fe69d36c1d5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:46:24.311313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d191d5d7-00a5-4d78-8ea2-fe69d36c1d5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:46:24.371755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191d5d7-00a5-4d78-8ea2-fe69d36c1d5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:46:24.373200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d191d5d7-00a5-4d78-8ea2-fe69d36c1d5c service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:46:24.382597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1197/2415] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:46:24 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:46:28.242906 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=6 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:46:28.634715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0947321-fd6e-4a26-9ebb-24677d920cad service swift] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:46:28.644005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0947321-fd6e-4a26-9ebb-24677d920cad service swift] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:46:28.647792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85714e-99ee-4c35-9141-e5cda7a11094 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:46:28.648372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85714e-99ee-4c35-9141-e5cda7a11094 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:46:28.648645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85714e-99ee-4c35-9141-e5cda7a11094 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:46:28.648947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85714e-99ee-4c35-9141-e5cda7a11094 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:46:28.657470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85714e-99ee-4c35-9141-e5cda7a11094 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:46:28.658870 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85714e-99ee-4c35-9141-e5cda7a11094 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:46:28.668566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1219/2416] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:46:28 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:46:34.378390 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=53 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:46:38.664316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=6 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:47:09.365448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c2fa8-b114-46ff-9649-f62cf8de59d7 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:47:09.366027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c2fa8-b114-46ff-9649-f62cf8de59d7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:47:09.366027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76c2fa8-b114-46ff-9649-f62cf8de59d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:47:09.385334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-f76c2fa8-b114-46ff-9649-f62cf8de59d7 None None] MFA Rules not processed for user `2945d87f1c114dd0b42c4e58a8133e76`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:47:09.386510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f76c2fa8-b114-46ff-9649-f62cf8de59d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:47:09.440591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1198/2417] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:47:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:47:16.091742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85714e-99ee-4c35-9141-e5cda7a11094 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:47:16.093292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8f85714e-99ee-4c35-9141-e5cda7a11094 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:47:16.160668 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85714e-99ee-4c35-9141-e5cda7a11094 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:47:16.165975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93139f81-07c6-4747-9697-7f3069bfff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:47:16.166602 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93139f81-07c6-4747-9697-7f3069bfff14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:47:16.166882 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93139f81-07c6-4747-9697-7f3069bfff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:47:16.167154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93139f81-07c6-4747-9697-7f3069bfff14 admin admin] PATH_INFO: `/v3/users/81d8036279ef4384bdbfa8c37e46a1d6` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:47:16.168268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93139f81-07c6-4747-9697-7f3069bfff14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81d8036279ef4384bdbfa8c37e46a1d6)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:47:16.170217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93139f81-07c6-4747-9697-7f3069bfff14 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:47:16.217102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-93139f81-07c6-4747-9697-7f3069bfff14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81d8036279ef4384bdbfa8c37e46a1d6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:47:16.225325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-93139f81-07c6-4747-9697-7f3069bfff14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81d8036279ef4384bdbfa8c37e46a1d6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:47:16.225982 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-93139f81-07c6-4747-9697-7f3069bfff14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81d8036279ef4384bdbfa8c37e46a1d6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:47:16.237812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-93139f81-07c6-4747-9697-7f3069bfff14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81d8036279ef4384bdbfa8c37e46a1d6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:47:16.245187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-93139f81-07c6-4747-9697-7f3069bfff14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81d8036279ef4384bdbfa8c37e46a1d6'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:47:16.250916 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1220/2418] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:47:16 2022] DELETE /identity/v3/users/81d8036279ef4384bdbfa8c37e46a1d6 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:47:16.422641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c2fa8-b114-46ff-9649-f62cf8de59d7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:47:16.434064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76c2fa8-b114-46ff-9649-f62cf8de59d7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:47:16.498606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:47:16.499232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:47:16.499506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:47:16.499773 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] PATH_INFO: `/v3/projects/3ebbea5186774c59942358c1fc284fa7` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:47:16.500897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ebbea5186774c59942358c1fc284fa7)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:47:16.502780 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:47:16.549340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ebbea5186774c59942358c1fc284fa7', 'request_id': 'req-9909e49f-7171-490a-b3ce-5ffa6d621b89'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:47:16.556127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ebbea5186774c59942358c1fc284fa7', 'request_id': 'req-9909e49f-7171-490a-b3ce-5ffa6d621b89'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:47:16.563529 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ebbea5186774c59942358c1fc284fa7', 'request_id': 'req-9909e49f-7171-490a-b3ce-5ffa6d621b89'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:47:16.574798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] The token cache is being invalidate because project 3ebbea5186774c59942358c1fc284fa7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:47:16.575099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:47:16.575751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9909e49f-7171-490a-b3ce-5ffa6d621b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:47:16.577370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1199/2419] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:47:16 2022] DELETE /identity/v3/projects/3ebbea5186774c59942358c1fc284fa7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:47:26.247560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:47:26.571816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:47:27.136711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc7ca6-40b6-46b2-9902-3b1e8eba0687 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:47:27.137099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc7ca6-40b6-46b2-9902-3b1e8eba0687 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:47:27.137383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dc7ca6-40b6-46b2-9902-3b1e8eba0687 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:47:27.158509 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-07dc7ca6-40b6-46b2-9902-3b1e8eba0687 None None] MFA Rules not processed for user `e305e7d0cbfc4aeca6164685d0155475`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:47:27.159728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-07dc7ca6-40b6-46b2-9902-3b1e8eba0687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:47:27.213537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1221/2420] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:47:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:47:37.206100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=25 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:47:37.479406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f3a8a-a086-427f-ab13-4e05de6c913e None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:47:37.479907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f3a8a-a086-427f-ab13-4e05de6c913e None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:47:37.480144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1f3a8a-a086-427f-ab13-4e05de6c913e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:47:37.505976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-cc1f3a8a-a086-427f-ab13-4e05de6c913e None None] MFA Rules not processed for user `4dd9f0bff3044d7db561dff21ac0e7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:47:37.507826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-cc1f3a8a-a086-427f-ab13-4e05de6c913e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:47:37.579748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1200/2421] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:47:37 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:47:39.444481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc7ca6-40b6-46b2-9902-3b1e8eba0687 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:47:39.446546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-07dc7ca6-40b6-46b2-9902-3b1e8eba0687 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:47:39.513064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dc7ca6-40b6-46b2-9902-3b1e8eba0687 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:47:39.516677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afc0ba-63a4-484c-8dd7-faa798756d4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:47:39.517372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afc0ba-63a4-484c-8dd7-faa798756d4f service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:47:39.517712 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afc0ba-63a4-484c-8dd7-faa798756d4f service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:47:39.518044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82afc0ba-63a4-484c-8dd7-faa798756d4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:47:39.520526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-82afc0ba-63a4-484c-8dd7-faa798756d4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:47:39.597344 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afc0ba-63a4-484c-8dd7-faa798756d4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:47:39.598756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82afc0ba-63a4-484c-8dd7-faa798756d4f service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:47:39.609943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1222/2422] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:47:39 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:47:43.362978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f3a8a-a086-427f-ab13-4e05de6c913e None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:47:43.373155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1f3a8a-a086-427f-ab13-4e05de6c913e None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:47:43.376690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1614f2b-263a-443a-9dc4-1cc027e8a294 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:47:43.377313 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1614f2b-263a-443a-9dc4-1cc027e8a294 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:47:43.377587 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1614f2b-263a-443a-9dc4-1cc027e8a294 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:47:43.377854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1614f2b-263a-443a-9dc4-1cc027e8a294 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:47:43.379555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b1614f2b-263a-443a-9dc4-1cc027e8a294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:47:43.393120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1614f2b-263a-443a-9dc4-1cc027e8a294 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:47:43.394533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1614f2b-263a-443a-9dc4-1cc027e8a294 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:47:43.406439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1201/2423] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:47:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:47:49.605152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=54 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:47:52.794024 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afc0ba-63a4-484c-8dd7-faa798756d4f service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:47:52.795526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-82afc0ba-63a4-484c-8dd7-faa798756d4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:47:52.856536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82afc0ba-63a4-484c-8dd7-faa798756d4f service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:47:52.860167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a68878-ef95-4d17-915d-ebd5621664da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:47:52.861367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a68878-ef95-4d17-915d-ebd5621664da service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:47:52.861655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a68878-ef95-4d17-915d-ebd5621664da service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:47:52.861924 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a68878-ef95-4d17-915d-ebd5621664da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:47:52.864121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-60a68878-ef95-4d17-915d-ebd5621664da service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:47:52.925720 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a68878-ef95-4d17-915d-ebd5621664da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:47:52.927840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a68878-ef95-4d17-915d-ebd5621664da service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:47:52.937347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1223/2424] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:47:52 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:47:53.401496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=31 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:02.932806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=48 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:13.355940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1614f2b-263a-443a-9dc4-1cc027e8a294 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:13.357353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b1614f2b-263a-443a-9dc4-1cc027e8a294 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:13.368662 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1614f2b-263a-443a-9dc4-1cc027e8a294 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:13.372218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cb1608-e0ad-4ee8-b477-04fa7193f647 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:13.372828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb1608-e0ad-4ee8-b477-04fa7193f647 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:13.373108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb1608-e0ad-4ee8-b477-04fa7193f647 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:13.373378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cb1608-e0ad-4ee8-b477-04fa7193f647 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:13.375021 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-03cb1608-e0ad-4ee8-b477-04fa7193f647 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:13.437017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb1608-e0ad-4ee8-b477-04fa7193f647 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:13.438439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cb1608-e0ad-4ee8-b477-04fa7193f647 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:13.447854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1202/2425] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:16.561473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ccd1-8021-4a8c-a56d-b115a7bb71a2 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:16.561942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ccd1-8021-4a8c-a56d-b115a7bb71a2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:16.562037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ccd1-8021-4a8c-a56d-b115a7bb71a2 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:16.563708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1224/2426] 10.222.0.39 () {54 vars in 961 bytes} [Tue Sep 6 22:48:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:48:16.569222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4db41a-ca1c-40a0-8b95-2910fd442148 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:16.569665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4db41a-ca1c-40a0-8b95-2910fd442148 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:16.569775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4db41a-ca1c-40a0-8b95-2910fd442148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:16.593394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-dc4db41a-ca1c-40a0-8b95-2910fd442148 None None] MFA Rules not processed for user `1d7e1b7dfa32448a88e0c90f10cdb2e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:16.594621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dc4db41a-ca1c-40a0-8b95-2910fd442148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:16.629007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998ccd1-8021-4a8c-a56d-b115a7bb71a2 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:16.630603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b998ccd1-8021-4a8c-a56d-b115a7bb71a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:16.648381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998ccd1-8021-4a8c-a56d-b115a7bb71a2 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:16.653813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17246987-4ba0-406a-9b71-51b88bfb2e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:16.654729 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246987-4ba0-406a-9b71-51b88bfb2e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:16.655007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246987-4ba0-406a-9b71-51b88bfb2e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:16.655280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17246987-4ba0-406a-9b71-51b88bfb2e44 admin admin] PATH_INFO: `/v3/users/2945d87f1c114dd0b42c4e58a8133e76` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:16.656499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17246987-4ba0-406a-9b71-51b88bfb2e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2945d87f1c114dd0b42c4e58a8133e76)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:16.658394 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17246987-4ba0-406a-9b71-51b88bfb2e44 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:16.692417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1203/2427] 10.222.0.39 () {58 vars in 1058 bytes} [Tue Sep 6 22:48:16 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:16.701529 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4db41a-ca1c-40a0-8b95-2910fd442148 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:16.711701 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4db41a-ca1c-40a0-8b95-2910fd442148 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:16.713701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-17246987-4ba0-406a-9b71-51b88bfb2e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2945d87f1c114dd0b42c4e58a8133e76'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:16.719260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a789f9-73f1-48f6-b847-f1a07d1f2177 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:16.719260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a789f9-73f1-48f6-b847-f1a07d1f2177 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:16.719260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a789f9-73f1-48f6-b847-f1a07d1f2177 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:16.719260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a789f9-73f1-48f6-b847-f1a07d1f2177 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:16.719984 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c0a789f9-73f1-48f6-b847-f1a07d1f2177 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:16.723998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-17246987-4ba0-406a-9b71-51b88bfb2e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2945d87f1c114dd0b42c4e58a8133e76'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:16.724667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-17246987-4ba0-406a-9b71-51b88bfb2e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2945d87f1c114dd0b42c4e58a8133e76'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:16.732271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a789f9-73f1-48f6-b847-f1a07d1f2177 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:16.733931 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a789f9-73f1-48f6-b847-f1a07d1f2177 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:16.736983 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-17246987-4ba0-406a-9b71-51b88bfb2e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2945d87f1c114dd0b42c4e58a8133e76'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:16.744418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1204/2428] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:16.744565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-17246987-4ba0-406a-9b71-51b88bfb2e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2945d87f1c114dd0b42c4e58a8133e76'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:16.750563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1225/2429] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:48:16 2022] DELETE /identity/v3/users/2945d87f1c114dd0b42c4e58a8133e76 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:16.931090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a789f9-73f1-48f6-b847-f1a07d1f2177 service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:16.941606 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a789f9-73f1-48f6-b847-f1a07d1f2177 service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:17.119596 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:17.120095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.120365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.120631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] PATH_INFO: `/v3/projects/84c39ea86193481fa08158197629ea85` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.121774 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] RBAC: Authorizing `identity:delete_project(project_id=84c39ea86193481fa08158197629ea85)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:17.123637 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:17.145888 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:17.173346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84c39ea86193481fa08158197629ea85', 'request_id': 'req-e391e329-3b2f-4312-bdc1-3521df2f1eea'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:17.180538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84c39ea86193481fa08158197629ea85', 'request_id': 'req-e391e329-3b2f-4312-bdc1-3521df2f1eea'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:17.188720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84c39ea86193481fa08158197629ea85', 'request_id': 'req-e391e329-3b2f-4312-bdc1-3521df2f1eea'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:17.200622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] The token cache is being invalidate because project 84c39ea86193481fa08158197629ea85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:48:17.200982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:17.201605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e391e329-3b2f-4312-bdc1-3521df2f1eea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:17.203102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1205/2430] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:48:16 2022] DELETE /identity/v3/projects/84c39ea86193481fa08158197629ea85 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:48:17.207933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab1ee0-93d0-4e7a-a90d-545b0ddcdb3b None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.208218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab1ee0-93d0-4e7a-a90d-545b0ddcdb3b None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.208494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ab1ee0-93d0-4e7a-a90d-545b0ddcdb3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.228034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-10ab1ee0-93d0-4e7a-a90d-545b0ddcdb3b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:17.229310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-10ab1ee0-93d0-4e7a-a90d-545b0ddcdb3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:17.287797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1226/2431] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:17.295514 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6829-83a1-45c5-bd60-eb36bded3c67 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.295791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6829-83a1-45c5-bd60-eb36bded3c67 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.296055 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6829-83a1-45c5-bd60-eb36bded3c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.315515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d1fb6829-83a1-45c5-bd60-eb36bded3c67 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:17.316695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d1fb6829-83a1-45c5-bd60-eb36bded3c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:17.325152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1206/2432] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:17.329435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab1ee0-93d0-4e7a-a90d-545b0ddcdb3b None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:17.339112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ab1ee0-93d0-4e7a-a90d-545b0ddcdb3b None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:17.343624 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805f762-8021-4ebb-9c61-c40e28004aa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:17.344215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805f762-8021-4ebb-9c61-c40e28004aa4 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.344492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805f762-8021-4ebb-9c61-c40e28004aa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.344840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805f762-8021-4ebb-9c61-c40e28004aa4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.345424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805f762-8021-4ebb-9c61-c40e28004aa4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:48:17.345725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805f762-8021-4ebb-9c61-c40e28004aa4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:17.347297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805f762-8021-4ebb-9c61-c40e28004aa4 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:17.358912 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1227/2433] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:48:17.363183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb6829-83a1-45c5-bd60-eb36bded3c67 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:17.372810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb6829-83a1-45c5-bd60-eb36bded3c67 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:17.377273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b56419-90a9-468a-bd0a-355bdf5fff1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:17.377849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b56419-90a9-468a-bd0a-355bdf5fff1b admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.378150 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b56419-90a9-468a-bd0a-355bdf5fff1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.378415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b56419-90a9-468a-bd0a-355bdf5fff1b admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.379038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b56419-90a9-468a-bd0a-355bdf5fff1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:17.380953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b56419-90a9-468a-bd0a-355bdf5fff1b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:17.408505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1207/2434] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:48:17 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:17.412700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805f762-8021-4ebb-9c61-c40e28004aa4 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:17.422558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805f762-8021-4ebb-9c61-c40e28004aa4 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:17.477679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9617b-65c5-47be-99fc-abc09f59d9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:17.478273 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9617b-65c5-47be-99fc-abc09f59d9ed admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.478548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9617b-65c5-47be-99fc-abc09f59d9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.478818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d9617b-65c5-47be-99fc-abc09f59d9ed admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.479472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9617b-65c5-47be-99fc-abc09f59d9ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:17.481409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d9617b-65c5-47be-99fc-abc09f59d9ed admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:17.489217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:17.504226 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1228/2435] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:48:17 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:17.508395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b56419-90a9-468a-bd0a-355bdf5fff1b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:17.518625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b56419-90a9-468a-bd0a-355bdf5fff1b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:17.523770 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c64c3e-089e-47d3-9efc-123a93ae454e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:17.524386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c64c3e-089e-47d3-9efc-123a93ae454e admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.524681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c64c3e-089e-47d3-9efc-123a93ae454e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.525229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c64c3e-089e-47d3-9efc-123a93ae454e admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.525852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c64c3e-089e-47d3-9efc-123a93ae454e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:17.527416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c64c3e-089e-47d3-9efc-123a93ae454e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:17.537481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1208/2436] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:17.541193 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9617b-65c5-47be-99fc-abc09f59d9ed admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:17.541342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c64c3e-089e-47d3-9efc-123a93ae454e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:17.542799 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d4c64c3e-089e-47d3-9efc-123a93ae454e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:17.551802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d9617b-65c5-47be-99fc-abc09f59d9ed admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:17.556825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831899-d8a3-4ee6-9951-2073ca5a3207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:17.557586 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831899-d8a3-4ee6-9951-2073ca5a3207 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.558212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831899-d8a3-4ee6-9951-2073ca5a3207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.558486 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5831899-d8a3-4ee6-9951-2073ca5a3207 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.559126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5831899-d8a3-4ee6-9951-2073ca5a3207 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:17.560735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5831899-d8a3-4ee6-9951-2073ca5a3207 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:17.575404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1229/2437] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:17.577373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831899-d8a3-4ee6-9951-2073ca5a3207 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:17.588080 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5831899-d8a3-4ee6-9951-2073ca5a3207 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:17.592965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b05ee-9ef8-4de3-a80c-1b2a7c393c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:17.593634 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b05ee-9ef8-4de3-a80c-1b2a7c393c0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.593914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b05ee-9ef8-4de3-a80c-1b2a7c393c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.594391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b05ee-9ef8-4de3-a80c-1b2a7c393c0a admin admin] PATH_INFO: `/v3/projects/f3a310b5cbbd45998ee9018833bc6c36/users/eb4fa59e775f45499a50c934e9fdae92/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.609286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b05ee-9ef8-4de3-a80c-1b2a7c393c0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3a310b5cbbd45998ee9018833bc6c36, user_id=eb4fa59e775f45499a50c934e9fdae92, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:17.611915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b05ee-9ef8-4de3-a80c-1b2a7c393c0a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:17.616338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c64c3e-089e-47d3-9efc-123a93ae454e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:17.622557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a162f96-e886-407a-bd9a-1269eadfb59c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:17.623166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a162f96-e886-407a-bd9a-1269eadfb59c service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.623440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a162f96-e886-407a-bd9a-1269eadfb59c service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.623709 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a162f96-e886-407a-bd9a-1269eadfb59c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.624962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1230/2438] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:17 2022] PUT /identity/v3/projects/f3a310b5cbbd45998ee9018833bc6c36/users/eb4fa59e775f45499a50c934e9fdae92/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:17.625601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8a162f96-e886-407a-bd9a-1269eadfb59c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:17.637285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c713d-b2de-439c-868d-a8b764e8ac2d None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.637563 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c713d-b2de-439c-868d-a8b764e8ac2d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.637832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556c713d-b2de-439c-868d-a8b764e8ac2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.667183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-556c713d-b2de-439c-868d-a8b764e8ac2d None None] MFA Rules not processed for user `eb4fa59e775f45499a50c934e9fdae92`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:17.668408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-556c713d-b2de-439c-868d-a8b764e8ac2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:17.690068 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a162f96-e886-407a-bd9a-1269eadfb59c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:17.692124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a162f96-e886-407a-bd9a-1269eadfb59c service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:17.704718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1209/2439] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:17 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:48:17.736017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1231/2440] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:17 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:48:17.740517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a162f96-e886-407a-bd9a-1269eadfb59c service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:17.747687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:17.752651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a162f96-e886-407a-bd9a-1269eadfb59c service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:17.813445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89b30-b951-4b9c-8fd8-39b8e051e55e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:17.814056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89b30-b951-4b9c-8fd8-39b8e051e55e admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.814332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89b30-b951-4b9c-8fd8-39b8e051e55e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.814599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b89b30-b951-4b9c-8fd8-39b8e051e55e admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.815216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b89b30-b951-4b9c-8fd8-39b8e051e55e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:17.817150 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b89b30-b951-4b9c-8fd8-39b8e051e55e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:17.844920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1210/2441] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:48:17 2022] POST /identity/v3/projects => generated 371 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:17.849095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c713d-b2de-439c-868d-a8b764e8ac2d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:17.859942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556c713d-b2de-439c-868d-a8b764e8ac2d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:17.926784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855c753-862b-4941-af65-5670be4503c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:17.927391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855c753-862b-4941-af65-5670be4503c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.927667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855c753-862b-4941-af65-5670be4503c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.927930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d855c753-862b-4941-af65-5670be4503c1 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.928547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855c753-862b-4941-af65-5670be4503c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:17.931299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d855c753-862b-4941-af65-5670be4503c1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:17.956782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1232/2442] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:48:17 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:48:17.961431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89b30-b951-4b9c-8fd8-39b8e051e55e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:17.971758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b89b30-b951-4b9c-8fd8-39b8e051e55e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:17.977920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1680bb0-43d9-4eaf-a358-5ae1c60387c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:17.978598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1680bb0-43d9-4eaf-a358-5ae1c60387c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:17.978944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1680bb0-43d9-4eaf-a358-5ae1c60387c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:17.979268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1680bb0-43d9-4eaf-a358-5ae1c60387c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:17.979971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1680bb0-43d9-4eaf-a358-5ae1c60387c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:17.981660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1680bb0-43d9-4eaf-a358-5ae1c60387c8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:17.993436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1211/2443] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:48:17.997579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855c753-862b-4941-af65-5670be4503c1 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:18.014989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d855c753-862b-4941-af65-5670be4503c1 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:18.020861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff8c38-a00f-45a3-90a7-539e8f504fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:18.020861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff8c38-a00f-45a3-90a7-539e8f504fff admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:18.021209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff8c38-a00f-45a3-90a7-539e8f504fff admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:18.021423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ff8c38-a00f-45a3-90a7-539e8f504fff admin admin] PATH_INFO: `/v3/projects/042fff90619c40fcbacad2f6f7c7da20/users/7004f79314a942149bdbc37045d49132/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:18.038691 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff8c38-a00f-45a3-90a7-539e8f504fff admin admin] RBAC: Authorizing `identity:create_grant(project_id=042fff90619c40fcbacad2f6f7c7da20, user_id=7004f79314a942149bdbc37045d49132, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:18.041527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ff8c38-a00f-45a3-90a7-539e8f504fff admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:18.055974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1233/2444] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:17 2022] PUT /identity/v3/projects/042fff90619c40fcbacad2f6f7c7da20/users/7004f79314a942149bdbc37045d49132/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:48:18.059977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1680bb0-43d9-4eaf-a358-5ae1c60387c8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:18.071219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1680bb0-43d9-4eaf-a358-5ae1c60387c8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:18.126482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542ec7ef-acef-472a-acea-8e212a075ae2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:18.127000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ec7ef-acef-472a-acea-8e212a075ae2 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:18.127272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ec7ef-acef-472a-acea-8e212a075ae2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:18.127536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542ec7ef-acef-472a-acea-8e212a075ae2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:18.128160 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ec7ef-acef-472a-acea-8e212a075ae2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:18.129749 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542ec7ef-acef-472a-acea-8e212a075ae2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:18.139737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1212/2445] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:18.143909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff8c38-a00f-45a3-90a7-539e8f504fff admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:18.154358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ff8c38-a00f-45a3-90a7-539e8f504fff admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:18.158871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd9f2b-03a3-487a-9c8f-818f2d4ada1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:18.159501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd9f2b-03a3-487a-9c8f-818f2d4ada1e admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:18.159774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd9f2b-03a3-487a-9c8f-818f2d4ada1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:18.160045 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcd9f2b-03a3-487a-9c8f-818f2d4ada1e admin admin] PATH_INFO: `/v3/projects/042fff90619c40fcbacad2f6f7c7da20/users/7004f79314a942149bdbc37045d49132/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:18.161936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd9f2b-03a3-487a-9c8f-818f2d4ada1e admin admin] RBAC: Authorizing `identity:create_grant(project_id=042fff90619c40fcbacad2f6f7c7da20, user_id=7004f79314a942149bdbc37045d49132, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:18.164558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcd9f2b-03a3-487a-9c8f-818f2d4ada1e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:18.167516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:INSERT=9,keystone:SELECT=91 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:18.175284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1234/2446] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:18 2022] PUT /identity/v3/projects/042fff90619c40fcbacad2f6f7c7da20/users/7004f79314a942149bdbc37045d49132/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:18.186222 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a2857-4d77-485c-a459-402e74d9a0b3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:18.186727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a2857-4d77-485c-a459-402e74d9a0b3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:18.187193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096a2857-4d77-485c-a459-402e74d9a0b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:18.216979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-096a2857-4d77-485c-a459-402e74d9a0b3 None None] MFA Rules not processed for user `7004f79314a942149bdbc37045d49132`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:18.218193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-096a2857-4d77-485c-a459-402e74d9a0b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:18.283185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:18.291602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1213/2447] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:18.300408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd9f2b-03a3-487a-9c8f-818f2d4ada1e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:18.302313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1dcd9f2b-03a3-487a-9c8f-818f2d4ada1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:18.359845 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcd9f2b-03a3-487a-9c8f-818f2d4ada1e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:18.363485 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9268cd8-315f-4d13-af7f-84f7565f4e96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:18.364084 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9268cd8-315f-4d13-af7f-84f7565f4e96 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:18.364364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9268cd8-315f-4d13-af7f-84f7565f4e96 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:18.364663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9268cd8-315f-4d13-af7f-84f7565f4e96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:18.373768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9268cd8-315f-4d13-af7f-84f7565f4e96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:18.375187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9268cd8-315f-4d13-af7f-84f7565f4e96 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:18.385239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1235/2448] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:18 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:48:18.771794 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a2857-4d77-485c-a459-402e74d9a0b3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:18.832220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096a2857-4d77-485c-a459-402e74d9a0b3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:18.836771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:18.837396 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:18.837673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:18.837946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] PATH_INFO: `/v3/users/eb4fa59e775f45499a50c934e9fdae92` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:18.839049 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb4fa59e775f45499a50c934e9fdae92)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:18.840972 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:18.885310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb4fa59e775f45499a50c934e9fdae92'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:18.885942 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb4fa59e775f45499a50c934e9fdae92'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:18.893166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb4fa59e775f45499a50c934e9fdae92'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:18.900517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb4fa59e775f45499a50c934e9fdae92'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:18.912502 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb4fa59e775f45499a50c934e9fdae92'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:18.918516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1214/2449] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:48:18 2022] DELETE /identity/v3/users/eb4fa59e775f45499a50c934e9fdae92 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:18.922984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf1f9c-4e35-43dc-b273-54b58ee1858f None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:18.923460 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf1f9c-4e35-43dc-b273-54b58ee1858f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:18.923560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bf1f9c-4e35-43dc-b273-54b58ee1858f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:18.942728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-71bf1f9c-4e35-43dc-b273-54b58ee1858f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:18.943908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-71bf1f9c-4e35-43dc-b273-54b58ee1858f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:19.000548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1236/2450] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:19.011061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:19.012418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:19.070555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb792c1d-2037-4c72-b261-24b864fa3f3e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:19.074813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f92bc-420d-4396-84a5-f33b7e27390d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:19.075430 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f92bc-420d-4396-84a5-f33b7e27390d service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:19.075705 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f92bc-420d-4396-84a5-f33b7e27390d service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:19.076126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0f92bc-420d-4396-84a5-f33b7e27390d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:19.085399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f92bc-420d-4396-84a5-f33b7e27390d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:19.086845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0f92bc-420d-4396-84a5-f33b7e27390d service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:19.096124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1215/2451] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:19.359183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf1f9c-4e35-43dc-b273-54b58ee1858f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:19.360672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-71bf1f9c-4e35-43dc-b273-54b58ee1858f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:19.373202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bf1f9c-4e35-43dc-b273-54b58ee1858f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:19.377813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:19.378441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:19.378721 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:19.378993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] PATH_INFO: `/v3/projects/f3a310b5cbbd45998ee9018833bc6c36` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:19.380118 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3a310b5cbbd45998ee9018833bc6c36)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:19.382050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:19.431782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3a310b5cbbd45998ee9018833bc6c36', 'request_id': 'req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:19.438975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3a310b5cbbd45998ee9018833bc6c36', 'request_id': 'req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:19.444335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3a310b5cbbd45998ee9018833bc6c36', 'request_id': 'req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:19.455560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] The token cache is being invalidate because project f3a310b5cbbd45998ee9018833bc6c36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:48:19.455845 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:19.456465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:19.457993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1237/2452] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:48:19 2022] DELETE /identity/v3/projects/f3a310b5cbbd45998ee9018833bc6c36 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:19.462478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f92bc-420d-4396-84a5-f33b7e27390d service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:19.471856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0f92bc-420d-4396-84a5-f33b7e27390d service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:19.489993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:19.531177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c18f6-8c85-48b8-a6f3-de3edb39cb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:19.532046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c18f6-8c85-48b8-a6f3-de3edb39cb44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:19.532456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c18f6-8c85-48b8-a6f3-de3edb39cb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:19.532891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c18f6-8c85-48b8-a6f3-de3edb39cb44 admin admin] PATH_INFO: `/v3/users/7004f79314a942149bdbc37045d49132` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:19.534125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c18f6-8c85-48b8-a6f3-de3edb39cb44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7004f79314a942149bdbc37045d49132)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:19.536039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2c18f6-8c85-48b8-a6f3-de3edb39cb44 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:19.584379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4f2c18f6-8c85-48b8-a6f3-de3edb39cb44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7004f79314a942149bdbc37045d49132'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:19.585052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4f2c18f6-8c85-48b8-a6f3-de3edb39cb44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7004f79314a942149bdbc37045d49132'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:19.593104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4f2c18f6-8c85-48b8-a6f3-de3edb39cb44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7004f79314a942149bdbc37045d49132'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:19.599788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4f2c18f6-8c85-48b8-a6f3-de3edb39cb44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7004f79314a942149bdbc37045d49132'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:19.612100 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4f2c18f6-8c85-48b8-a6f3-de3edb39cb44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7004f79314a942149bdbc37045d49132'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:19.618028 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1216/2453] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:48:19 2022] DELETE /identity/v3/users/7004f79314a942149bdbc37045d49132 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:19.905103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:19.906609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:19.920228 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708e02a2-aa17-46f3-bb2f-edb0e21e59a1 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:19.940474 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:19.980071 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:19.980619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:19.980941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:19.981216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] PATH_INFO: `/v3/projects/042fff90619c40fcbacad2f6f7c7da20` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:19.982327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=042fff90619c40fcbacad2f6f7c7da20)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:19.984221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:20.035194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '042fff90619c40fcbacad2f6f7c7da20', 'request_id': 'req-b460d36d-5238-4a0f-aa64-290da0ffc241'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:20.042470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '042fff90619c40fcbacad2f6f7c7da20', 'request_id': 'req-b460d36d-5238-4a0f-aa64-290da0ffc241'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:20.048052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '042fff90619c40fcbacad2f6f7c7da20', 'request_id': 'req-b460d36d-5238-4a0f-aa64-290da0ffc241'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:20.059847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] The token cache is being invalidate because project 042fff90619c40fcbacad2f6f7c7da20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:48:20.060126 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:20.060770 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b460d36d-5238-4a0f-aa64-290da0ffc241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:20.062497 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1238/2454] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:48:19 2022] DELETE /identity/v3/projects/042fff90619c40fcbacad2f6f7c7da20 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:48:20.067717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9209c5b-3a7e-4669-99a4-8d3bec3586a6 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.068167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9209c5b-3a7e-4669-99a4-8d3bec3586a6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.068410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9209c5b-3a7e-4669-99a4-8d3bec3586a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.089352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d9209c5b-3a7e-4669-99a4-8d3bec3586a6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:20.090654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d9209c5b-3a7e-4669-99a4-8d3bec3586a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:20.152252 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1217/2455] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:20.160351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b3479-b97c-4548-bb5f-e43cc0aa9fc4 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.160686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b3479-b97c-4548-bb5f-e43cc0aa9fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.160999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b3479-b97c-4548-bb5f-e43cc0aa9fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.181727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-451b3479-b97c-4548-bb5f-e43cc0aa9fc4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:20.182931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-451b3479-b97c-4548-bb5f-e43cc0aa9fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:20.190888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1239/2456] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:48:20.195817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9209c5b-3a7e-4669-99a4-8d3bec3586a6 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:20.205115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9209c5b-3a7e-4669-99a4-8d3bec3586a6 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:20.210025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714aca6-d4c1-449f-bab9-0c7d41bb49e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:20.211295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714aca6-d4c1-449f-bab9-0c7d41bb49e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.211850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714aca6-d4c1-449f-bab9-0c7d41bb49e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.211992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3714aca6-d4c1-449f-bab9-0c7d41bb49e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.212520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714aca6-d4c1-449f-bab9-0c7d41bb49e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:48:20.212853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714aca6-d4c1-449f-bab9-0c7d41bb49e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:20.214459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3714aca6-d4c1-449f-bab9-0c7d41bb49e3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:20.226153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1218/2457] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:48:20.231032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b3479-b97c-4548-bb5f-e43cc0aa9fc4 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:20.240328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b3479-b97c-4548-bb5f-e43cc0aa9fc4 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:20.245589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366c67a-f186-46d9-93e5-c3ff6f28fb6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:20.246169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366c67a-f186-46d9-93e5-c3ff6f28fb6b admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.246445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366c67a-f186-46d9-93e5-c3ff6f28fb6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.246714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c366c67a-f186-46d9-93e5-c3ff6f28fb6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.247329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366c67a-f186-46d9-93e5-c3ff6f28fb6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:20.249242 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c366c67a-f186-46d9-93e5-c3ff6f28fb6b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:20.280205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1240/2458] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:48:20 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:48:20.284343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714aca6-d4c1-449f-bab9-0c7d41bb49e3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:20.295033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3714aca6-d4c1-449f-bab9-0c7d41bb49e3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:20.339446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:20.372598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e92f0-0def-4f72-a193-fab9065a6057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:20.373239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e92f0-0def-4f72-a193-fab9065a6057 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.373515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e92f0-0def-4f72-a193-fab9065a6057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.373788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4e92f0-0def-4f72-a193-fab9065a6057 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.374413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e92f0-0def-4f72-a193-fab9065a6057 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:20.376306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4e92f0-0def-4f72-a193-fab9065a6057 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:20.402154 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1219/2459] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:48:20 2022] POST /identity/v3/users => generated 283 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:20.404231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366c67a-f186-46d9-93e5-c3ff6f28fb6b admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:20.416040 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c366c67a-f186-46d9-93e5-c3ff6f28fb6b admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:20.420929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bbac8-e500-4e7d-8613-73c7f50a91b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:20.421535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bbac8-e500-4e7d-8613-73c7f50a91b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.421806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bbac8-e500-4e7d-8613-73c7f50a91b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.422068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bbac8-e500-4e7d-8613-73c7f50a91b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.422701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bbac8-e500-4e7d-8613-73c7f50a91b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:20.424258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bbac8-e500-4e7d-8613-73c7f50a91b5 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:20.435595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1241/2460] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:20.439741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e92f0-0def-4f72-a193-fab9065a6057 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:20.450375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4e92f0-0def-4f72-a193-fab9065a6057 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:20.454930 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c1814-fd2a-4d36-8774-c69fc6fbac49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:20.455528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c1814-fd2a-4d36-8774-c69fc6fbac49 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.455797 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c1814-fd2a-4d36-8774-c69fc6fbac49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.456062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c1814-fd2a-4d36-8774-c69fc6fbac49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.456727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c1814-fd2a-4d36-8774-c69fc6fbac49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:20.458310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c1814-fd2a-4d36-8774-c69fc6fbac49 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:20.468580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1220/2461] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:48:20.472733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bbac8-e500-4e7d-8613-73c7f50a91b5 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:20.483334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bbac8-e500-4e7d-8613-73c7f50a91b5 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:20.487909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed3838-8418-433c-96bb-eed0717a8666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:20.488549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed3838-8418-433c-96bb-eed0717a8666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.488851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed3838-8418-433c-96bb-eed0717a8666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.489122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deed3838-8418-433c-96bb-eed0717a8666 admin admin] PATH_INFO: `/v3/projects/b04306bcdaef4c6c8945cebe59df28ba/users/435906ccb09e4f82a807ec66bb4d3774/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.504066 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed3838-8418-433c-96bb-eed0717a8666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b04306bcdaef4c6c8945cebe59df28ba, user_id=435906ccb09e4f82a807ec66bb4d3774, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:20.506763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deed3838-8418-433c-96bb-eed0717a8666 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:20.520217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1242/2462] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:20 2022] PUT /identity/v3/projects/b04306bcdaef4c6c8945cebe59df28ba/users/435906ccb09e4f82a807ec66bb4d3774/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:48:20.531991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6d85d-fb98-4f6c-aae1-1e26de616a6f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.532354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6d85d-fb98-4f6c-aae1-1e26de616a6f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.532626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b6d85d-fb98-4f6c-aae1-1e26de616a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.561675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-16b6d85d-fb98-4f6c-aae1-1e26de616a6f None None] MFA Rules not processed for user `435906ccb09e4f82a807ec66bb4d3774`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:20.562859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-16b6d85d-fb98-4f6c-aae1-1e26de616a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:20.627840 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1221/2463] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:20.632849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed3838-8418-433c-96bb-eed0717a8666 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:20.681527 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:20.697151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deed3838-8418-433c-96bb-eed0717a8666 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:20.701755 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9242f-f051-49ab-917c-fd5c3f643c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:20.702348 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9242f-f051-49ab-917c-fd5c3f643c97 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.702626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9242f-f051-49ab-917c-fd5c3f643c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.702896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9242f-f051-49ab-917c-fd5c3f643c97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.703538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9242f-f051-49ab-917c-fd5c3f643c97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:20.705454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9242f-f051-49ab-917c-fd5c3f643c97 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:20.732055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1243/2464] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:48:20 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:20.736164 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6d85d-fb98-4f6c-aae1-1e26de616a6f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:20.747180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b6d85d-fb98-4f6c-aae1-1e26de616a6f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:20.802898 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e074683-7f15-42c7-a157-0d1a332dbed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:20.803485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e074683-7f15-42c7-a157-0d1a332dbed3 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.803760 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e074683-7f15-42c7-a157-0d1a332dbed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.804029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e074683-7f15-42c7-a157-0d1a332dbed3 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.805250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e074683-7f15-42c7-a157-0d1a332dbed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:20.807137 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e074683-7f15-42c7-a157-0d1a332dbed3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:20.832237 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1222/2465] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:48:20 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:20.837187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9242f-f051-49ab-917c-fd5c3f643c97 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:20.847250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9242f-f051-49ab-917c-fd5c3f643c97 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:20.852150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccb63f-a77b-44ce-aeeb-e63d141dce00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:20.852732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccb63f-a77b-44ce-aeeb-e63d141dce00 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.853033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccb63f-a77b-44ce-aeeb-e63d141dce00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.853302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ccb63f-a77b-44ce-aeeb-e63d141dce00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.853925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ccb63f-a77b-44ce-aeeb-e63d141dce00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:20.855496 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ccb63f-a77b-44ce-aeeb-e63d141dce00 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:20.865792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1244/2466] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:20.870436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e074683-7f15-42c7-a157-0d1a332dbed3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:20.880163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e074683-7f15-42c7-a157-0d1a332dbed3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:20.885786 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9871719-093b-4cb5-bb1e-d81cb77e9331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:20.886422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9871719-093b-4cb5-bb1e-d81cb77e9331 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.886695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9871719-093b-4cb5-bb1e-d81cb77e9331 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.886961 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9871719-093b-4cb5-bb1e-d81cb77e9331 admin admin] PATH_INFO: `/v3/projects/e174914274f147f8b06a4e7a969f27ca/users/83a81f1ee63044b58159d71644e92c1c/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.902528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9871719-093b-4cb5-bb1e-d81cb77e9331 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e174914274f147f8b06a4e7a969f27ca, user_id=83a81f1ee63044b58159d71644e92c1c, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:20.905328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9871719-093b-4cb5-bb1e-d81cb77e9331 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:20.917838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1223/2467] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:20 2022] PUT /identity/v3/projects/e174914274f147f8b06a4e7a969f27ca/users/83a81f1ee63044b58159d71644e92c1c/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:20.921815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccb63f-a77b-44ce-aeeb-e63d141dce00 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:20.932553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ccb63f-a77b-44ce-aeeb-e63d141dce00 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:20.987217 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c332370-eabc-48eb-b49d-36343fca0638 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:20.987963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c332370-eabc-48eb-b49d-36343fca0638 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:20.988085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c332370-eabc-48eb-b49d-36343fca0638 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:20.988262 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c332370-eabc-48eb-b49d-36343fca0638 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:20.988918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c332370-eabc-48eb-b49d-36343fca0638 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:20.990494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c332370-eabc-48eb-b49d-36343fca0638 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:21.000886 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1245/2468] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:20 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:21.005205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9871719-093b-4cb5-bb1e-d81cb77e9331 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:21.011230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:21.017093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9871719-093b-4cb5-bb1e-d81cb77e9331 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:21.022842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6116a40-a31d-4059-ac9a-0570e7b028ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:21.023617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6116a40-a31d-4059-ac9a-0570e7b028ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:21.023895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6116a40-a31d-4059-ac9a-0570e7b028ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:21.024164 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6116a40-a31d-4059-ac9a-0570e7b028ca admin admin] PATH_INFO: `/v3/projects/e174914274f147f8b06a4e7a969f27ca/users/83a81f1ee63044b58159d71644e92c1c/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:21.026354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6116a40-a31d-4059-ac9a-0570e7b028ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=e174914274f147f8b06a4e7a969f27ca, user_id=83a81f1ee63044b58159d71644e92c1c, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:21.029019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6116a40-a31d-4059-ac9a-0570e7b028ca admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:21.039106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1224/2469] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:21 2022] PUT /identity/v3/projects/e174914274f147f8b06a4e7a969f27ca/users/83a81f1ee63044b58159d71644e92c1c/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:21.049336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510c1c1-e6c3-4f02-ad42-2e89fbedf043 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:21.049625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510c1c1-e6c3-4f02-ad42-2e89fbedf043 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:21.049898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d510c1c1-e6c3-4f02-ad42-2e89fbedf043 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:21.079063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d510c1c1-e6c3-4f02-ad42-2e89fbedf043 None None] MFA Rules not processed for user `83a81f1ee63044b58159d71644e92c1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:21.080261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d510c1c1-e6c3-4f02-ad42-2e89fbedf043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:21.146242 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1246/2470] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:21.157878 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6116a40-a31d-4059-ac9a-0570e7b028ca admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:21.159579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d6116a40-a31d-4059-ac9a-0570e7b028ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:21.215137 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6116a40-a31d-4059-ac9a-0570e7b028ca admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:21.218630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25a38a-fa7f-4fba-ae02-869daed15757 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:21.219208 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a38a-fa7f-4fba-ae02-869daed15757 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:21.219476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a38a-fa7f-4fba-ae02-869daed15757 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:21.219764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a38a-fa7f-4fba-ae02-869daed15757 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:21.228370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25a38a-fa7f-4fba-ae02-869daed15757 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:21.229824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c25a38a-fa7f-4fba-ae02-869daed15757 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:21.238872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1225/2471] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:21.661659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510c1c1-e6c3-4f02-ad42-2e89fbedf043 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:21.712170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:21.730410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d510c1c1-e6c3-4f02-ad42-2e89fbedf043 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:21.735072 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:21.735705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:21.735981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:21.736251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] PATH_INFO: `/v3/users/435906ccb09e4f82a807ec66bb4d3774` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:21.737399 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] RBAC: Authorizing `identity:delete_user(user_id=435906ccb09e4f82a807ec66bb4d3774)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:21.739309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:21.785350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '435906ccb09e4f82a807ec66bb4d3774'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:21.793118 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '435906ccb09e4f82a807ec66bb4d3774'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:21.793748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '435906ccb09e4f82a807ec66bb4d3774'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:21.805455 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '435906ccb09e4f82a807ec66bb4d3774'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:21.812267 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '435906ccb09e4f82a807ec66bb4d3774'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:21.817999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1247/2472] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:48:21 2022] DELETE /identity/v3/users/435906ccb09e4f82a807ec66bb4d3774 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:21.822683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f845227-2bb0-4d93-8820-8719488fcce4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:21.822964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f845227-2bb0-4d93-8820-8719488fcce4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:21.823234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f845227-2bb0-4d93-8820-8719488fcce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:21.846659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-5f845227-2bb0-4d93-8820-8719488fcce4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:21.847940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5f845227-2bb0-4d93-8820-8719488fcce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:21.908016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1226/2473] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:21.918580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:21.919965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:21.978106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df899ff-0b6e-4a14-bf70-03f314ddcebd admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:21.981793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8e387-67c2-4285-9947-e1c1b7ee4450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:21.982393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e387-67c2-4285-9947-e1c1b7ee4450 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:21.982669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e387-67c2-4285-9947-e1c1b7ee4450 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:21.982941 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e387-67c2-4285-9947-e1c1b7ee4450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:21.992249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8e387-67c2-4285-9947-e1c1b7ee4450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:21.993710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb8e387-67c2-4285-9947-e1c1b7ee4450 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:22.003004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1248/2474] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:48:22.269827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f845227-2bb0-4d93-8820-8719488fcce4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:22.271295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5f845227-2bb0-4d93-8820-8719488fcce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:22.285581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f845227-2bb0-4d93-8820-8719488fcce4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:22.291712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:22.292557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:22.292869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:22.293147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] PATH_INFO: `/v3/projects/b04306bcdaef4c6c8945cebe59df28ba` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:22.294280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=b04306bcdaef4c6c8945cebe59df28ba)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:22.296177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:22.344012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b04306bcdaef4c6c8945cebe59df28ba', 'request_id': 'req-88db1cca-c5a1-4757-9538-23b7f5a960cf'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.350274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b04306bcdaef4c6c8945cebe59df28ba', 'request_id': 'req-88db1cca-c5a1-4757-9538-23b7f5a960cf'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.357734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b04306bcdaef4c6c8945cebe59df28ba', 'request_id': 'req-88db1cca-c5a1-4757-9538-23b7f5a960cf'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.369224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] The token cache is being invalidate because project b04306bcdaef4c6c8945cebe59df28ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:48:22.369536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.370154 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.371611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1227/2475] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:48:22 2022] DELETE /identity/v3/projects/b04306bcdaef4c6c8945cebe59df28ba => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:22.375572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8e387-67c2-4285-9947-e1c1b7ee4450 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:22.385208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb8e387-67c2-4285-9947-e1c1b7ee4450 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:22.442536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d802d9b2-fd9b-4dac-b6ee-61fdd0ac939b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:22.443159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802d9b2-fd9b-4dac-b6ee-61fdd0ac939b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:22.443431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802d9b2-fd9b-4dac-b6ee-61fdd0ac939b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:22.443698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d802d9b2-fd9b-4dac-b6ee-61fdd0ac939b admin admin] PATH_INFO: `/v3/users/83a81f1ee63044b58159d71644e92c1c` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:22.444830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802d9b2-fd9b-4dac-b6ee-61fdd0ac939b admin admin] RBAC: Authorizing `identity:delete_user(user_id=83a81f1ee63044b58159d71644e92c1c)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:22.446952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d802d9b2-fd9b-4dac-b6ee-61fdd0ac939b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:22.452129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:22.494131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d802d9b2-fd9b-4dac-b6ee-61fdd0ac939b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83a81f1ee63044b58159d71644e92c1c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.501312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d802d9b2-fd9b-4dac-b6ee-61fdd0ac939b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83a81f1ee63044b58159d71644e92c1c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.501926 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d802d9b2-fd9b-4dac-b6ee-61fdd0ac939b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83a81f1ee63044b58159d71644e92c1c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.514641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d802d9b2-fd9b-4dac-b6ee-61fdd0ac939b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83a81f1ee63044b58159d71644e92c1c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.523887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d802d9b2-fd9b-4dac-b6ee-61fdd0ac939b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83a81f1ee63044b58159d71644e92c1c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.532798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1249/2476] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:48:22 2022] DELETE /identity/v3/users/83a81f1ee63044b58159d71644e92c1c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:22.804464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:22.806000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:22.822242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88db1cca-c5a1-4757-9538-23b7f5a960cf admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:22.838315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:22.887422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:22.887963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:22.888235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:22.888499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] PATH_INFO: `/v3/projects/e174914274f147f8b06a4e7a969f27ca` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:22.889627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e174914274f147f8b06a4e7a969f27ca)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:22.891496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:22.936927 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e174914274f147f8b06a4e7a969f27ca', 'request_id': 'req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.942958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e174914274f147f8b06a4e7a969f27ca', 'request_id': 'req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.950909 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e174914274f147f8b06a4e7a969f27ca', 'request_id': 'req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.961415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] The token cache is being invalidate because project e174914274f147f8b06a4e7a969f27ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:48:22.961700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.962323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-bf91b021-e6a2-43b1-8ca3-cd85d04e186b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:22.963787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1228/2477] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:48:22 2022] DELETE /identity/v3/projects/e174914274f147f8b06a4e7a969f27ca => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:22.968866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86176f9-f7dc-4a0d-99ac-66582ae53557 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:22.969148 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86176f9-f7dc-4a0d-99ac-66582ae53557 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:22.969419 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86176f9-f7dc-4a0d-99ac-66582ae53557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:22.989740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e86176f9-f7dc-4a0d-99ac-66582ae53557 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:22.990930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e86176f9-f7dc-4a0d-99ac-66582ae53557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:23.049421 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1250/2478] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:23.057299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a0cb5-ef39-4015-9c0b-551cd510ccef None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.057579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a0cb5-ef39-4015-9c0b-551cd510ccef None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.057849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3a0cb5-ef39-4015-9c0b-551cd510ccef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.078368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3b3a0cb5-ef39-4015-9c0b-551cd510ccef None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:23.079564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3b3a0cb5-ef39-4015-9c0b-551cd510ccef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:23.088002 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1229/2479] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:23.093641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86176f9-f7dc-4a0d-99ac-66582ae53557 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.103376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86176f9-f7dc-4a0d-99ac-66582ae53557 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.108020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfd844-b26c-4fb3-b744-8d3087d14c5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.108615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd844-b26c-4fb3-b744-8d3087d14c5f admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.108936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd844-b26c-4fb3-b744-8d3087d14c5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.109211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddfd844-b26c-4fb3-b744-8d3087d14c5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.109844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfd844-b26c-4fb3-b744-8d3087d14c5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:48:23.110144 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfd844-b26c-4fb3-b744-8d3087d14c5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.111730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddfd844-b26c-4fb3-b744-8d3087d14c5f admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.123200 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1251/2480] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:48:23.127481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a0cb5-ef39-4015-9c0b-551cd510ccef None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.137231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3a0cb5-ef39-4015-9c0b-551cd510ccef None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.142062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fcd27-20a4-442e-b24b-5cf91b05ffe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.142595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fcd27-20a4-442e-b24b-5cf91b05ffe3 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.142872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fcd27-20a4-442e-b24b-5cf91b05ffe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.143146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72fcd27-20a4-442e-b24b-5cf91b05ffe3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.143758 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fcd27-20a4-442e-b24b-5cf91b05ffe3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.145689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72fcd27-20a4-442e-b24b-5cf91b05ffe3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.173279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1230/2481] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:48:23 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:23.177403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfd844-b26c-4fb3-b744-8d3087d14c5f admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.187919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddfd844-b26c-4fb3-b744-8d3087d14c5f admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.243213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca4941-d3b0-49aa-addc-c313025ce6b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.243808 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca4941-d3b0-49aa-addc-c313025ce6b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.244083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca4941-d3b0-49aa-addc-c313025ce6b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.244351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ca4941-d3b0-49aa-addc-c313025ce6b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.245003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca4941-d3b0-49aa-addc-c313025ce6b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.246891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ca4941-d3b0-49aa-addc-c313025ce6b2 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.266940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1252/2482] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:48:23 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:23.271125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fcd27-20a4-442e-b24b-5cf91b05ffe3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.282370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72fcd27-20a4-442e-b24b-5cf91b05ffe3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.287027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0008efa-f053-484d-bb5d-01f8be601f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.287618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0008efa-f053-484d-bb5d-01f8be601f84 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.287895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0008efa-f053-484d-bb5d-01f8be601f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.288508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0008efa-f053-484d-bb5d-01f8be601f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.289703 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0008efa-f053-484d-bb5d-01f8be601f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.291275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0008efa-f053-484d-bb5d-01f8be601f84 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.301033 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1231/2483] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:23.305060 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca4941-d3b0-49aa-addc-c313025ce6b2 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.310205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:23.316017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ca4941-d3b0-49aa-addc-c313025ce6b2 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.322356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563ee959-d3f7-4a3b-b312-699895a69bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.322950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ee959-d3f7-4a3b-b312-699895a69bee admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.323230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ee959-d3f7-4a3b-b312-699895a69bee admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.323505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563ee959-d3f7-4a3b-b312-699895a69bee admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.324142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ee959-d3f7-4a3b-b312-699895a69bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.325753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563ee959-d3f7-4a3b-b312-699895a69bee admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.336827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1253/2484] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:23.340838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0008efa-f053-484d-bb5d-01f8be601f84 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.350595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0008efa-f053-484d-bb5d-01f8be601f84 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.355747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c766c2-4fab-4d1b-9a00-3c793cdf61d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.356376 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c766c2-4fab-4d1b-9a00-3c793cdf61d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.356650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c766c2-4fab-4d1b-9a00-3c793cdf61d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.356944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c766c2-4fab-4d1b-9a00-3c793cdf61d3 admin admin] PATH_INFO: `/v3/projects/b6eac355d2df45dabaec744c840170d1/users/84b40b0156534be79eab3d87d4ffedc3/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.372190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c766c2-4fab-4d1b-9a00-3c793cdf61d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6eac355d2df45dabaec744c840170d1, user_id=84b40b0156534be79eab3d87d4ffedc3, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.374941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c766c2-4fab-4d1b-9a00-3c793cdf61d3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.387612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1232/2485] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:23 2022] PUT /identity/v3/projects/b6eac355d2df45dabaec744c840170d1/users/84b40b0156534be79eab3d87d4ffedc3/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:23.397601 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afb492-0207-4eb6-a6b5-df84033ce84c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.397843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afb492-0207-4eb6-a6b5-df84033ce84c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.398747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afb492-0207-4eb6-a6b5-df84033ce84c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.426021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-02afb492-0207-4eb6-a6b5-df84033ce84c None None] MFA Rules not processed for user `84b40b0156534be79eab3d87d4ffedc3`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:23.427211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-02afb492-0207-4eb6-a6b5-df84033ce84c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:23.488955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1254/2486] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:23.493993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c766c2-4fab-4d1b-9a00-3c793cdf61d3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.537190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:23.556611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c766c2-4fab-4d1b-9a00-3c793cdf61d3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.561229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17c780-26e5-4e5c-8c75-cce154a0a947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.561822 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c780-26e5-4e5c-8c75-cce154a0a947 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.562092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c780-26e5-4e5c-8c75-cce154a0a947 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.562354 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17c780-26e5-4e5c-8c75-cce154a0a947 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.562962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17c780-26e5-4e5c-8c75-cce154a0a947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.564920 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c17c780-26e5-4e5c-8c75-cce154a0a947 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.590397 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1233/2487] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:48:23 2022] POST /identity/v3/projects => generated 387 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:23.594456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afb492-0207-4eb6-a6b5-df84033ce84c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.605570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afb492-0207-4eb6-a6b5-df84033ce84c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.659884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76023e0c-f668-4140-b648-5e10a3d44e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.660398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76023e0c-f668-4140-b648-5e10a3d44e2d admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.660669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76023e0c-f668-4140-b648-5e10a3d44e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.660959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76023e0c-f668-4140-b648-5e10a3d44e2d admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.661566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76023e0c-f668-4140-b648-5e10a3d44e2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.663431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76023e0c-f668-4140-b648-5e10a3d44e2d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.686160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1255/2488] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:48:23 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:23.690159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17c780-26e5-4e5c-8c75-cce154a0a947 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.701347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17c780-26e5-4e5c-8c75-cce154a0a947 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.705908 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527a2d5-23e7-4493-9f0a-dad5ad8353fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.706489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527a2d5-23e7-4493-9f0a-dad5ad8353fb admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.706798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527a2d5-23e7-4493-9f0a-dad5ad8353fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.707070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527a2d5-23e7-4493-9f0a-dad5ad8353fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.707692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527a2d5-23e7-4493-9f0a-dad5ad8353fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.709337 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d527a2d5-23e7-4493-9f0a-dad5ad8353fb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.720536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1234/2489] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:23.724546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76023e0c-f668-4140-b648-5e10a3d44e2d admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.734567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76023e0c-f668-4140-b648-5e10a3d44e2d admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.739219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a28e7-e67f-4f8b-81f4-73f7a7876b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.739864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a28e7-e67f-4f8b-81f4-73f7a7876b7a admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.740143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a28e7-e67f-4f8b-81f4-73f7a7876b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.740418 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714a28e7-e67f-4f8b-81f4-73f7a7876b7a admin admin] PATH_INFO: `/v3/projects/3bc4cc36e08e451d8acaaffd9894d21f/users/dfbc9ff583c34c659eb6092f9937ff0b/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.754968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a28e7-e67f-4f8b-81f4-73f7a7876b7a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc4cc36e08e451d8acaaffd9894d21f, user_id=dfbc9ff583c34c659eb6092f9937ff0b, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.758393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714a28e7-e67f-4f8b-81f4-73f7a7876b7a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.771621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1256/2490] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:23 2022] PUT /identity/v3/projects/3bc4cc36e08e451d8acaaffd9894d21f/users/dfbc9ff583c34c659eb6092f9937ff0b/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:23.776394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527a2d5-23e7-4493-9f0a-dad5ad8353fb admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.786689 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527a2d5-23e7-4493-9f0a-dad5ad8353fb admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.842710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf048aad-7b0d-4ca8-9b5d-dee6b7d7961c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.843302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf048aad-7b0d-4ca8-9b5d-dee6b7d7961c admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.843580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf048aad-7b0d-4ca8-9b5d-dee6b7d7961c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.843853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf048aad-7b0d-4ca8-9b5d-dee6b7d7961c admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.844485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf048aad-7b0d-4ca8-9b5d-dee6b7d7961c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.846097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf048aad-7b0d-4ca8-9b5d-dee6b7d7961c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.855638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1235/2491] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:23.859765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a28e7-e67f-4f8b-81f4-73f7a7876b7a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:23.871824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714a28e7-e67f-4f8b-81f4-73f7a7876b7a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:23.876910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f97a9-15b6-4d99-816b-ffba3ee5d97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:23.877573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f97a9-15b6-4d99-816b-ffba3ee5d97d admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.877853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f97a9-15b6-4d99-816b-ffba3ee5d97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.878121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f97a9-15b6-4d99-816b-ffba3ee5d97d admin admin] PATH_INFO: `/v3/projects/3bc4cc36e08e451d8acaaffd9894d21f/users/dfbc9ff583c34c659eb6092f9937ff0b/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.880063 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f97a9-15b6-4d99-816b-ffba3ee5d97d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bc4cc36e08e451d8acaaffd9894d21f, user_id=dfbc9ff583c34c659eb6092f9937ff0b, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:23.883339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec8f97a9-15b6-4d99-816b-ffba3ee5d97d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:23.894041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1257/2492] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:23 2022] PUT /identity/v3/projects/3bc4cc36e08e451d8acaaffd9894d21f/users/dfbc9ff583c34c659eb6092f9937ff0b/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:23.905263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77acb97-d737-44da-9a8a-e0c39235b41d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:23.905544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77acb97-d737-44da-9a8a-e0c39235b41d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:23.905814 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77acb97-d737-44da-9a8a-e0c39235b41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:23.934697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b77acb97-d737-44da-9a8a-e0c39235b41d None None] MFA Rules not processed for user `dfbc9ff583c34c659eb6092f9937ff0b`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:23.935826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b77acb97-d737-44da-9a8a-e0c39235b41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:24.006062 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1236/2493] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:23 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:24.018029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f97a9-15b6-4d99-816b-ffba3ee5d97d admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:24.019745 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ec8f97a9-15b6-4d99-816b-ffba3ee5d97d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:24.060339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:24.082783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec8f97a9-15b6-4d99-816b-ffba3ee5d97d admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:24.087021 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed59a16-792e-416d-a392-3dca0c2ab53b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:24.087621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed59a16-792e-416d-a392-3dca0c2ab53b service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:24.087896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed59a16-792e-416d-a392-3dca0c2ab53b service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:24.088179 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed59a16-792e-416d-a392-3dca0c2ab53b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:24.096912 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed59a16-792e-416d-a392-3dca0c2ab53b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:24.098317 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed59a16-792e-416d-a392-3dca0c2ab53b service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:24.109528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1258/2494] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:24 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:24.175243 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77acb97-d737-44da-9a8a-e0c39235b41d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:24.216165 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:24.239043 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b77acb97-d737-44da-9a8a-e0c39235b41d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:24.243641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:24.244259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:24.244530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:24.244825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] PATH_INFO: `/v3/users/84b40b0156534be79eab3d87d4ffedc3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:24.245935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] RBAC: Authorizing `identity:delete_user(user_id=84b40b0156534be79eab3d87d4ffedc3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:24.247839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:24.293865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84b40b0156534be79eab3d87d4ffedc3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:24.294499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84b40b0156534be79eab3d87d4ffedc3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:24.302234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84b40b0156534be79eab3d87d4ffedc3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:24.310168 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84b40b0156534be79eab3d87d4ffedc3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:24.322919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84b40b0156534be79eab3d87d4ffedc3'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:24.328735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1237/2495] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:48:24 2022] DELETE /identity/v3/users/84b40b0156534be79eab3d87d4ffedc3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:24.333260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8aab7-a6e8-4c1a-bdf6-060c43e1fbef None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:24.333543 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8aab7-a6e8-4c1a-bdf6-060c43e1fbef None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:24.333819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8aab7-a6e8-4c1a-bdf6-060c43e1fbef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:24.354233 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-67e8aab7-a6e8-4c1a-bdf6-060c43e1fbef None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:24.355434 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-67e8aab7-a6e8-4c1a-bdf6-060c43e1fbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:24.413867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1259/2496] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:24.423647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:24.425076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:24.481959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e05e337-a987-44af-a4bd-32150f49a87c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:24.485480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec95bd-fad3-48a6-95cb-ceee47902c7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:24.486078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec95bd-fad3-48a6-95cb-ceee47902c7e service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:24.486353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec95bd-fad3-48a6-95cb-ceee47902c7e service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:24.486620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fec95bd-fad3-48a6-95cb-ceee47902c7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:24.504470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec95bd-fad3-48a6-95cb-ceee47902c7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:24.506285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fec95bd-fad3-48a6-95cb-ceee47902c7e service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:24.516997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1238/2497] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:24.783617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8aab7-a6e8-4c1a-bdf6-060c43e1fbef None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:24.785204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-67e8aab7-a6e8-4c1a-bdf6-060c43e1fbef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:24.799669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8aab7-a6e8-4c1a-bdf6-060c43e1fbef None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:24.806221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:24.806924 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:24.807281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:24.807606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] PATH_INFO: `/v3/projects/b6eac355d2df45dabaec744c840170d1` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:24.809298 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6eac355d2df45dabaec744c840170d1)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:24.812272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:24.860416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6eac355d2df45dabaec744c840170d1', 'request_id': 'req-0a340d24-55fa-4e14-898b-8ae12a320c72'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:24.867792 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6eac355d2df45dabaec744c840170d1', 'request_id': 'req-0a340d24-55fa-4e14-898b-8ae12a320c72'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:24.873198 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6eac355d2df45dabaec744c840170d1', 'request_id': 'req-0a340d24-55fa-4e14-898b-8ae12a320c72'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:24.884454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] The token cache is being invalidate because project b6eac355d2df45dabaec744c840170d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:48:24.884733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:24.885369 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:24.886821 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1260/2498] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:48:24 2022] DELETE /identity/v3/projects/b6eac355d2df45dabaec744c840170d1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:24.890720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec95bd-fad3-48a6-95cb-ceee47902c7e service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:24.900327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fec95bd-fad3-48a6-95cb-ceee47902c7e service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:24.955274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac32727-f6b2-4a1c-9041-43559cb93970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:24.955890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32727-f6b2-4a1c-9041-43559cb93970 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:24.956166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32727-f6b2-4a1c-9041-43559cb93970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:24.956438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac32727-f6b2-4a1c-9041-43559cb93970 admin admin] PATH_INFO: `/v3/users/dfbc9ff583c34c659eb6092f9937ff0b` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:24.957583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac32727-f6b2-4a1c-9041-43559cb93970 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfbc9ff583c34c659eb6092f9937ff0b)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:24.959485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac32727-f6b2-4a1c-9041-43559cb93970 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:24.960978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:25.004638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dac32727-f6b2-4a1c-9041-43559cb93970 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfbc9ff583c34c659eb6092f9937ff0b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:25.005350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dac32727-f6b2-4a1c-9041-43559cb93970 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfbc9ff583c34c659eb6092f9937ff0b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:25.014367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dac32727-f6b2-4a1c-9041-43559cb93970 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfbc9ff583c34c659eb6092f9937ff0b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:25.021239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dac32727-f6b2-4a1c-9041-43559cb93970 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfbc9ff583c34c659eb6092f9937ff0b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:25.033704 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-dac32727-f6b2-4a1c-9041-43559cb93970 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfbc9ff583c34c659eb6092f9937ff0b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:25.039889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1239/2499] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:48:24 2022] DELETE /identity/v3/users/dfbc9ff583c34c659eb6092f9937ff0b => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:25.390712 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:25.392207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:25.406236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a340d24-55fa-4e14-898b-8ae12a320c72 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:25.464552 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:25.465207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:25.465491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:25.465763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] PATH_INFO: `/v3/projects/3bc4cc36e08e451d8acaaffd9894d21f` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:25.466875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bc4cc36e08e451d8acaaffd9894d21f)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:25.468786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:25.470215 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:25.515923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bc4cc36e08e451d8acaaffd9894d21f', 'request_id': 'req-4843da4d-0d44-4666-9e47-8b4b6831e907'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:25.523511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bc4cc36e08e451d8acaaffd9894d21f', 'request_id': 'req-4843da4d-0d44-4666-9e47-8b4b6831e907'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:25.529743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bc4cc36e08e451d8acaaffd9894d21f', 'request_id': 'req-4843da4d-0d44-4666-9e47-8b4b6831e907'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:25.541615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] The token cache is being invalidate because project 3bc4cc36e08e451d8acaaffd9894d21f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:48:25.541863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:25.542494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4843da4d-0d44-4666-9e47-8b4b6831e907 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:25.543957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1261/2500] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:48:25 2022] DELETE /identity/v3/projects/3bc4cc36e08e451d8acaaffd9894d21f => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:25.548842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72488-ae46-4a6d-9732-66f9a20e498b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:25.549302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72488-ae46-4a6d-9732-66f9a20e498b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:25.549399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f72488-ae46-4a6d-9732-66f9a20e498b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:25.569353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-43f72488-ae46-4a6d-9732-66f9a20e498b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:25.570991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-43f72488-ae46-4a6d-9732-66f9a20e498b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:25.631437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1240/2501] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:25.641292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971e32-4660-4e57-b105-c5d545cd1261 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:25.641577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971e32-4660-4e57-b105-c5d545cd1261 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:25.641848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29971e32-4660-4e57-b105-c5d545cd1261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:25.661688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-29971e32-4660-4e57-b105-c5d545cd1261 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:25.662898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-29971e32-4660-4e57-b105-c5d545cd1261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:25.671219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1262/2502] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:25.675487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f72488-ae46-4a6d-9732-66f9a20e498b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:25.685662 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f72488-ae46-4a6d-9732-66f9a20e498b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:25.690199 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec357ea-3bea-49ba-913b-0506482aee04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:25.690784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec357ea-3bea-49ba-913b-0506482aee04 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:25.691057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec357ea-3bea-49ba-913b-0506482aee04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:25.691323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec357ea-3bea-49ba-913b-0506482aee04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:25.691965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec357ea-3bea-49ba-913b-0506482aee04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:48:25.692258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec357ea-3bea-49ba-913b-0506482aee04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:25.693859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec357ea-3bea-49ba-913b-0506482aee04 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:25.706369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1241/2503] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:48:25.710447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29971e32-4660-4e57-b105-c5d545cd1261 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:25.720172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29971e32-4660-4e57-b105-c5d545cd1261 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:25.724798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09715a0-4c9c-41c2-ad50-a9a6c21af7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:25.725416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09715a0-4c9c-41c2-ad50-a9a6c21af7f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:25.725692 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09715a0-4c9c-41c2-ad50-a9a6c21af7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:25.725963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09715a0-4c9c-41c2-ad50-a9a6c21af7f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:25.726591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09715a0-4c9c-41c2-ad50-a9a6c21af7f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:25.728467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09715a0-4c9c-41c2-ad50-a9a6c21af7f1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:25.755233 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1263/2504] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:48:25 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:48:25.760109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec357ea-3bea-49ba-913b-0506482aee04 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:25.769965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec357ea-3bea-49ba-913b-0506482aee04 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:25.826264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea78e7-315c-4c0e-92eb-6c1d43b4154c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:25.826791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea78e7-315c-4c0e-92eb-6c1d43b4154c admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:25.827066 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea78e7-315c-4c0e-92eb-6c1d43b4154c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:25.827332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ea78e7-315c-4c0e-92eb-6c1d43b4154c admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:25.827940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea78e7-315c-4c0e-92eb-6c1d43b4154c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:25.829864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ea78e7-315c-4c0e-92eb-6c1d43b4154c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:25.840603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:25.854445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1242/2505] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:48:25 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:25.855455 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09715a0-4c9c-41c2-ad50-a9a6c21af7f1 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:25.865567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09715a0-4c9c-41c2-ad50-a9a6c21af7f1 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:25.870756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcb123-e4c6-446f-81a5-90b005239f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:25.871341 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcb123-e4c6-446f-81a5-90b005239f8e admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:25.871612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcb123-e4c6-446f-81a5-90b005239f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:25.871893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afcb123-e4c6-446f-81a5-90b005239f8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:25.872506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afcb123-e4c6-446f-81a5-90b005239f8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:25.874106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afcb123-e4c6-446f-81a5-90b005239f8e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:25.883791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1264/2506] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:25.887561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea78e7-315c-4c0e-92eb-6c1d43b4154c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:25.897241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ea78e7-315c-4c0e-92eb-6c1d43b4154c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:25.902201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece0e0ed-da1c-4d96-9269-1e68ab5ebd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:25.902777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0e0ed-da1c-4d96-9269-1e68ab5ebd5f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:25.903050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0e0ed-da1c-4d96-9269-1e68ab5ebd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:25.903316 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece0e0ed-da1c-4d96-9269-1e68ab5ebd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:25.903936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0e0ed-da1c-4d96-9269-1e68ab5ebd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:25.905531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece0e0ed-da1c-4d96-9269-1e68ab5ebd5f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:25.916195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1243/2507] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:25.920287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcb123-e4c6-446f-81a5-90b005239f8e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:25.931265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afcb123-e4c6-446f-81a5-90b005239f8e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:25.935847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fda20-b6f7-40e2-982b-455c5a868ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:25.936490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fda20-b6f7-40e2-982b-455c5a868ec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:25.936790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fda20-b6f7-40e2-982b-455c5a868ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:25.937143 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86fda20-b6f7-40e2-982b-455c5a868ec2 admin admin] PATH_INFO: `/v3/projects/bcc1048ca96743bdaf11d8a1264a0541/users/4795a8fdbf8142bb86566062919d65d8/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:25.951630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fda20-b6f7-40e2-982b-455c5a868ec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcc1048ca96743bdaf11d8a1264a0541, user_id=4795a8fdbf8142bb86566062919d65d8, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:25.954303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86fda20-b6f7-40e2-982b-455c5a868ec2 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:25.966619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1265/2508] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:25 2022] PUT /identity/v3/projects/bcc1048ca96743bdaf11d8a1264a0541/users/4795a8fdbf8142bb86566062919d65d8/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:25.977380 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e7ff1-1e5b-4fc6-ad3c-ab95bf30a65c None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:25.977662 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e7ff1-1e5b-4fc6-ad3c-ab95bf30a65c None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:25.977935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1e7ff1-1e5b-4fc6-ad3c-ab95bf30a65c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:26.005564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4b1e7ff1-1e5b-4fc6-ad3c-ab95bf30a65c None None] MFA Rules not processed for user `4795a8fdbf8142bb86566062919d65d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:26.006824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4b1e7ff1-1e5b-4fc6-ad3c-ab95bf30a65c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:26.070638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1244/2509] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:26.075325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fda20-b6f7-40e2-982b-455c5a868ec2 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:26.135345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86fda20-b6f7-40e2-982b-455c5a868ec2 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:26.140027 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf593fd-c400-41cc-aadf-8380f6518e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:26.140650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf593fd-c400-41cc-aadf-8380f6518e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:26.140952 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf593fd-c400-41cc-aadf-8380f6518e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:26.141225 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf593fd-c400-41cc-aadf-8380f6518e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:26.141832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf593fd-c400-41cc-aadf-8380f6518e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:26.143701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf593fd-c400-41cc-aadf-8380f6518e64 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:26.170507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1266/2510] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:48:26 2022] POST /identity/v3/projects => generated 381 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:26.174776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e7ff1-1e5b-4fc6-ad3c-ab95bf30a65c None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:26.186259 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1e7ff1-1e5b-4fc6-ad3c-ab95bf30a65c None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:26.245710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b44e4f-0d9d-4377-9372-1c2edc3b9d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:26.246330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b44e4f-0d9d-4377-9372-1c2edc3b9d48 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:26.246604 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b44e4f-0d9d-4377-9372-1c2edc3b9d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:26.246867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b44e4f-0d9d-4377-9372-1c2edc3b9d48 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:26.247490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b44e4f-0d9d-4377-9372-1c2edc3b9d48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:26.249468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b44e4f-0d9d-4377-9372-1c2edc3b9d48 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:26.270189 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1245/2511] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:48:26 2022] POST /identity/v3/users => generated 297 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:26.275364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf593fd-c400-41cc-aadf-8380f6518e64 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:26.285606 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf593fd-c400-41cc-aadf-8380f6518e64 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:26.290761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea6d7b-b096-4500-ad8e-1fa0f4d1569e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:26.291351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea6d7b-b096-4500-ad8e-1fa0f4d1569e admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:26.291629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea6d7b-b096-4500-ad8e-1fa0f4d1569e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:26.292654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ea6d7b-b096-4500-ad8e-1fa0f4d1569e admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:26.293328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea6d7b-b096-4500-ad8e-1fa0f4d1569e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:26.294902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ea6d7b-b096-4500-ad8e-1fa0f4d1569e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:26.301589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:26.311500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1267/2512] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:26.312803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b44e4f-0d9d-4377-9372-1c2edc3b9d48 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:26.323927 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b44e4f-0d9d-4377-9372-1c2edc3b9d48 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:26.328541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b750-67d9-4e28-adea-aabf43855131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:26.329254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b750-67d9-4e28-adea-aabf43855131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:26.329529 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b750-67d9-4e28-adea-aabf43855131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:26.329802 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c5b750-67d9-4e28-adea-aabf43855131 admin admin] PATH_INFO: `/v3/projects/3014ac943bbc4306a5330d658a45b3f0/users/f81e0098bbdb4067b9f332929df4f93b/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:26.344839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5b750-67d9-4e28-adea-aabf43855131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3014ac943bbc4306a5330d658a45b3f0, user_id=f81e0098bbdb4067b9f332929df4f93b, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:26.347504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c5b750-67d9-4e28-adea-aabf43855131 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:26.363318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1246/2513] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:26 2022] PUT /identity/v3/projects/3014ac943bbc4306a5330d658a45b3f0/users/f81e0098bbdb4067b9f332929df4f93b/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:48:26.367809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea6d7b-b096-4500-ad8e-1fa0f4d1569e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:26.378040 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ea6d7b-b096-4500-ad8e-1fa0f4d1569e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:26.436390 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367326d2-813c-47de-85c3-e699e265b371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:26.437183 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367326d2-813c-47de-85c3-e699e265b371 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:26.437556 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367326d2-813c-47de-85c3-e699e265b371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:26.437936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367326d2-813c-47de-85c3-e699e265b371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:26.438677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367326d2-813c-47de-85c3-e699e265b371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:26.440483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367326d2-813c-47de-85c3-e699e265b371 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:26.462281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1268/2514] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:26 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:48:26.466261 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b750-67d9-4e28-adea-aabf43855131 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:26.477432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c5b750-67d9-4e28-adea-aabf43855131 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:26.481999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c00ef-19b5-43a3-b204-63d536d6bdd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:26.482638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c00ef-19b5-43a3-b204-63d536d6bdd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:26.482911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c00ef-19b5-43a3-b204-63d536d6bdd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:26.483176 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817c00ef-19b5-43a3-b204-63d536d6bdd3 admin admin] PATH_INFO: `/v3/projects/3014ac943bbc4306a5330d658a45b3f0/users/f81e0098bbdb4067b9f332929df4f93b/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:26.485072 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c00ef-19b5-43a3-b204-63d536d6bdd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3014ac943bbc4306a5330d658a45b3f0, user_id=f81e0098bbdb4067b9f332929df4f93b, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:26.487665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817c00ef-19b5-43a3-b204-63d536d6bdd3 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:26.498345 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1247/2515] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:26 2022] PUT /identity/v3/projects/3014ac943bbc4306a5330d658a45b3f0/users/f81e0098bbdb4067b9f332929df4f93b/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:26.508513 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dd55d-18f7-48ea-8eab-1252f06efc92 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:26.508827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dd55d-18f7-48ea-8eab-1252f06efc92 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:26.509104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939dd55d-18f7-48ea-8eab-1252f06efc92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:26.537525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-939dd55d-18f7-48ea-8eab-1252f06efc92 None None] MFA Rules not processed for user `f81e0098bbdb4067b9f332929df4f93b`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:26.539591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-939dd55d-18f7-48ea-8eab-1252f06efc92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:26.608764 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1269/2516] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:26.619352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c00ef-19b5-43a3-b204-63d536d6bdd3 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:26.621078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-817c00ef-19b5-43a3-b204-63d536d6bdd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:26.652391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:26.679992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817c00ef-19b5-43a3-b204-63d536d6bdd3 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:26.683736 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2763f-d5ff-4ad0-95b5-50c34c76978d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:26.684434 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2763f-d5ff-4ad0-95b5-50c34c76978d service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:26.684709 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2763f-d5ff-4ad0-95b5-50c34c76978d service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:26.685003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2763f-d5ff-4ad0-95b5-50c34c76978d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:26.694002 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2763f-d5ff-4ad0-95b5-50c34c76978d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:26.696052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2763f-d5ff-4ad0-95b5-50c34c76978d service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:26.705547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1248/2517] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:26.810828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dd55d-18f7-48ea-8eab-1252f06efc92 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:26.820837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939dd55d-18f7-48ea-8eab-1252f06efc92 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:26.824338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e83908-29af-46e6-948b-22e9ce04e8ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:26.824949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e83908-29af-46e6-948b-22e9ce04e8ee service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:26.825224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e83908-29af-46e6-948b-22e9ce04e8ee service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:26.825495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e83908-29af-46e6-948b-22e9ce04e8ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:26.879989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e83908-29af-46e6-948b-22e9ce04e8ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:26.881443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e83908-29af-46e6-948b-22e9ce04e8ee service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:26.890677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1270/2518] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:26.897090 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2763f-d5ff-4ad0-95b5-50c34c76978d service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:26.899272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3ae2763f-d5ff-4ad0-95b5-50c34c76978d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:26.973517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2763f-d5ff-4ad0-95b5-50c34c76978d service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:26.979390 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde812b3-4bc0-4d3b-a3d2-638f082e2a48 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:26.979984 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde812b3-4bc0-4d3b-a3d2-638f082e2a48 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:26.980256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde812b3-4bc0-4d3b-a3d2-638f082e2a48 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:26.980520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde812b3-4bc0-4d3b-a3d2-638f082e2a48 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:26.982269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bde812b3-4bc0-4d3b-a3d2-638f082e2a48 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:27.040950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde812b3-4bc0-4d3b-a3d2-638f082e2a48 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:27.042538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde812b3-4bc0-4d3b-a3d2-638f082e2a48 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:27.051741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1249/2519] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:26 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:29.497256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e83908-29af-46e6-948b-22e9ce04e8ee service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:29.518211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:29.563524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e83908-29af-46e6-948b-22e9ce04e8ee service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:29.568416 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:29.569078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:29.569353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:29.569620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] PATH_INFO: `/v3/users/4795a8fdbf8142bb86566062919d65d8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:29.570738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4795a8fdbf8142bb86566062919d65d8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:29.573333 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:29.620791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4795a8fdbf8142bb86566062919d65d8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:29.627965 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4795a8fdbf8142bb86566062919d65d8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:29.628617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4795a8fdbf8142bb86566062919d65d8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:29.640469 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4795a8fdbf8142bb86566062919d65d8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:29.647167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4795a8fdbf8142bb86566062919d65d8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:29.653395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1271/2520] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:48:29 2022] DELETE /identity/v3/users/4795a8fdbf8142bb86566062919d65d8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:29.657885 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14327a7-86e7-4bda-abb2-b3c3c0856f35 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:29.658345 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14327a7-86e7-4bda-abb2-b3c3c0856f35 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:29.658458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14327a7-86e7-4bda-abb2-b3c3c0856f35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:29.677563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a14327a7-86e7-4bda-abb2-b3c3c0856f35 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:29.678746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a14327a7-86e7-4bda-abb2-b3c3c0856f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:29.737705 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1250/2521] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:29.748028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:29.749435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:29.806898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cc2159-28a8-4065-94e6-bbe600c6d7d7 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:29.810545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed825a-a5d4-43fd-b46f-6ec1bb4b4c97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:29.811146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed825a-a5d4-43fd-b46f-6ec1bb4b4c97 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:29.811413 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed825a-a5d4-43fd-b46f-6ec1bb4b4c97 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:29.811731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed825a-a5d4-43fd-b46f-6ec1bb4b4c97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:29.820678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed825a-a5d4-43fd-b46f-6ec1bb4b4c97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:29.822230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed825a-a5d4-43fd-b46f-6ec1bb4b4c97 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:29.831730 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1272/2522] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:30.135302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14327a7-86e7-4bda-abb2-b3c3c0856f35 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:30.136682 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a14327a7-86e7-4bda-abb2-b3c3c0856f35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:30.149070 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14327a7-86e7-4bda-abb2-b3c3c0856f35 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:30.153919 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:30.155082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:30.155358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:30.155626 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] PATH_INFO: `/v3/projects/bcc1048ca96743bdaf11d8a1264a0541` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:30.156740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcc1048ca96743bdaf11d8a1264a0541)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:30.158660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:30.160285 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:30.205358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcc1048ca96743bdaf11d8a1264a0541', 'request_id': 'req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.211302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcc1048ca96743bdaf11d8a1264a0541', 'request_id': 'req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.219215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcc1048ca96743bdaf11d8a1264a0541', 'request_id': 'req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.229520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] The token cache is being invalidate because project bcc1048ca96743bdaf11d8a1264a0541 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:48:30.229808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.230438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.231947 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1251/2523] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:48:30 2022] DELETE /identity/v3/projects/bcc1048ca96743bdaf11d8a1264a0541 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:30.236111 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed825a-a5d4-43fd-b46f-6ec1bb4b4c97 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:30.246483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed825a-a5d4-43fd-b46f-6ec1bb4b4c97 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:30.287181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:30.307135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc8ae86-19e8-4be3-9922-df7b3e496692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:30.307680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8ae86-19e8-4be3-9922-df7b3e496692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:30.307951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8ae86-19e8-4be3-9922-df7b3e496692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:30.308218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc8ae86-19e8-4be3-9922-df7b3e496692 admin admin] PATH_INFO: `/v3/users/f81e0098bbdb4067b9f332929df4f93b` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:30.309342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8ae86-19e8-4be3-9922-df7b3e496692 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f81e0098bbdb4067b9f332929df4f93b)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:30.311259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc8ae86-19e8-4be3-9922-df7b3e496692 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:30.357382 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8bc8ae86-19e8-4be3-9922-df7b3e496692 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f81e0098bbdb4067b9f332929df4f93b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.364381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8bc8ae86-19e8-4be3-9922-df7b3e496692 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f81e0098bbdb4067b9f332929df4f93b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.365024 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8bc8ae86-19e8-4be3-9922-df7b3e496692 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f81e0098bbdb4067b9f332929df4f93b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.376899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8bc8ae86-19e8-4be3-9922-df7b3e496692 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f81e0098bbdb4067b9f332929df4f93b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.384336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8bc8ae86-19e8-4be3-9922-df7b3e496692 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f81e0098bbdb4067b9f332929df4f93b'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.390276 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1273/2524] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:48:30 2022] DELETE /identity/v3/users/f81e0098bbdb4067b9f332929df4f93b => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:30.659853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:30.661346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:30.674536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcbba29-86e5-4d36-a1d6-9aad91e0e02c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:30.731157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:30.731777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:30.732052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:30.732320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] PATH_INFO: `/v3/projects/3014ac943bbc4306a5330d658a45b3f0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:30.733441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3014ac943bbc4306a5330d658a45b3f0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:30.735326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:30.780918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3014ac943bbc4306a5330d658a45b3f0', 'request_id': 'req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.786932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3014ac943bbc4306a5330d658a45b3f0', 'request_id': 'req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.794490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3014ac943bbc4306a5330d658a45b3f0', 'request_id': 'req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.804418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] The token cache is being invalidate because project 3014ac943bbc4306a5330d658a45b3f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:48:30.804700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.805345 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b1ca2dd0-c8ae-4435-a0fa-63e73fefb065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:30.807805 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1252/2525] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:48:30 2022] DELETE /identity/v3/projects/3014ac943bbc4306a5330d658a45b3f0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:30.812661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fef6c-0c7b-49c9-af05-b4ea3b63aa38 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:30.813150 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fef6c-0c7b-49c9-af05-b4ea3b63aa38 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:30.813268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275fef6c-0c7b-49c9-af05-b4ea3b63aa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:30.833866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-275fef6c-0c7b-49c9-af05-b4ea3b63aa38 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:30.834996 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-275fef6c-0c7b-49c9-af05-b4ea3b63aa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:30.904557 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1274/2526] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:30.913197 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de95b1-119d-4e52-b1bc-34aa90503240 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:30.913476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de95b1-119d-4e52-b1bc-34aa90503240 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:30.913747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de95b1-119d-4e52-b1bc-34aa90503240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:30.933865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-85de95b1-119d-4e52-b1bc-34aa90503240 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:30.935045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-85de95b1-119d-4e52-b1bc-34aa90503240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:30.942853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1253/2527] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:30.947827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fef6c-0c7b-49c9-af05-b4ea3b63aa38 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:30.957458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275fef6c-0c7b-49c9-af05-b4ea3b63aa38 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:30.962117 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785915c-a090-46fc-85f5-b4d1d3bc3407 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:30.962688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785915c-a090-46fc-85f5-b4d1d3bc3407 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:30.962962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785915c-a090-46fc-85f5-b4d1d3bc3407 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:30.963232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4785915c-a090-46fc-85f5-b4d1d3bc3407 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:30.963864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785915c-a090-46fc-85f5-b4d1d3bc3407 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:48:30.964161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785915c-a090-46fc-85f5-b4d1d3bc3407 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:30.965769 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4785915c-a090-46fc-85f5-b4d1d3bc3407 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:30.976998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1275/2528] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:48:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:48:30.981255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de95b1-119d-4e52-b1bc-34aa90503240 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:30.990314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de95b1-119d-4e52-b1bc-34aa90503240 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:30.995811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e631486-529c-4ae5-a7b2-16c48634e39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:30.996408 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e631486-529c-4ae5-a7b2-16c48634e39e admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:30.996685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e631486-529c-4ae5-a7b2-16c48634e39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:30.997004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e631486-529c-4ae5-a7b2-16c48634e39e admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:30.997668 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e631486-529c-4ae5-a7b2-16c48634e39e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:30.999565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e631486-529c-4ae5-a7b2-16c48634e39e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:31.011441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:31.028707 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1254/2529] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:48:30 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:31.032910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785915c-a090-46fc-85f5-b4d1d3bc3407 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:31.043680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4785915c-a090-46fc-85f5-b4d1d3bc3407 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:31.087783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:31.100674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde55cf9-ceab-4d7b-b496-a03b41f21958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:31.101310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde55cf9-ceab-4d7b-b496-a03b41f21958 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:31.101587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde55cf9-ceab-4d7b-b496-a03b41f21958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:31.101858 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde55cf9-ceab-4d7b-b496-a03b41f21958 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:31.103269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde55cf9-ceab-4d7b-b496-a03b41f21958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:31.105199 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde55cf9-ceab-4d7b-b496-a03b41f21958 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:31.131791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1276/2530] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:48:31 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:31.133002 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e631486-529c-4ae5-a7b2-16c48634e39e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:31.143355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e631486-529c-4ae5-a7b2-16c48634e39e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:31.148084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46e67f-5f99-408e-9eb7-720410739328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:31.148687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46e67f-5f99-408e-9eb7-720410739328 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:31.149000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46e67f-5f99-408e-9eb7-720410739328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:31.149269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa46e67f-5f99-408e-9eb7-720410739328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:31.149913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46e67f-5f99-408e-9eb7-720410739328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:31.151761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa46e67f-5f99-408e-9eb7-720410739328 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:31.162163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1255/2531] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:48:31.166924 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde55cf9-ceab-4d7b-b496-a03b41f21958 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:31.177017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde55cf9-ceab-4d7b-b496-a03b41f21958 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:31.181919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3141559-ab48-4876-8a64-07f2d6a100f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:31.182523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3141559-ab48-4876-8a64-07f2d6a100f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:31.182862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3141559-ab48-4876-8a64-07f2d6a100f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:31.183138 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3141559-ab48-4876-8a64-07f2d6a100f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:31.183831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3141559-ab48-4876-8a64-07f2d6a100f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:31.185464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3141559-ab48-4876-8a64-07f2d6a100f5 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:31.199798 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1277/2532] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:48:31.204094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46e67f-5f99-408e-9eb7-720410739328 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:31.215506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa46e67f-5f99-408e-9eb7-720410739328 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:31.223377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198974d-3bb4-4dac-8039-cb4534cc31b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:31.224284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198974d-3bb4-4dac-8039-cb4534cc31b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:31.224574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198974d-3bb4-4dac-8039-cb4534cc31b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:31.224881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a198974d-3bb4-4dac-8039-cb4534cc31b6 admin admin] PATH_INFO: `/v3/projects/b0ff502834994518adbe6eb1f8876be8/users/0f6ff15ef5a34e1ab53ceb9f613d9071/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:31.240213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198974d-3bb4-4dac-8039-cb4534cc31b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ff502834994518adbe6eb1f8876be8, user_id=0f6ff15ef5a34e1ab53ceb9f613d9071, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:31.243057 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a198974d-3bb4-4dac-8039-cb4534cc31b6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:31.256874 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1256/2533] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:31 2022] PUT /identity/v3/projects/b0ff502834994518adbe6eb1f8876be8/users/0f6ff15ef5a34e1ab53ceb9f613d9071/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:31.267815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297d405-94fe-4cb6-835e-41ccb388628f None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:31.268099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297d405-94fe-4cb6-835e-41ccb388628f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:31.268374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297d405-94fe-4cb6-835e-41ccb388628f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:31.301259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-3297d405-94fe-4cb6-835e-41ccb388628f None None] MFA Rules not processed for user `0f6ff15ef5a34e1ab53ceb9f613d9071`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:31.302452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3297d405-94fe-4cb6-835e-41ccb388628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:31.365205 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1278/2534] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:31.376833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198974d-3bb4-4dac-8039-cb4534cc31b6 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:31.378518 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a198974d-3bb4-4dac-8039-cb4534cc31b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:31.435977 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a198974d-3bb4-4dac-8039-cb4534cc31b6 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:31.439585 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9610b48f-7cbf-4916-8a3b-8087bc54a37a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:31.440175 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610b48f-7cbf-4916-8a3b-8087bc54a37a service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:31.440447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610b48f-7cbf-4916-8a3b-8087bc54a37a service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:31.440713 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9610b48f-7cbf-4916-8a3b-8087bc54a37a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:31.449414 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610b48f-7cbf-4916-8a3b-8087bc54a37a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:31.450818 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9610b48f-7cbf-4916-8a3b-8087bc54a37a service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:31.459497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1257/2535] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:41.362383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=44,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:41.455393 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:44.309779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297d405-94fe-4cb6-835e-41ccb388628f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:44.311253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3297d405-94fe-4cb6-835e-41ccb388628f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:44.407829 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297d405-94fe-4cb6-835e-41ccb388628f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:44.413212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9aeb70-e487-4311-a6b3-e470db4b2c85 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:44.413813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9aeb70-e487-4311-a6b3-e470db4b2c85 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:44.414087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9aeb70-e487-4311-a6b3-e470db4b2c85 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:44.414357 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9aeb70-e487-4311-a6b3-e470db4b2c85 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:44.416167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ba9aeb70-e487-4311-a6b3-e470db4b2c85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:44.427780 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9aeb70-e487-4311-a6b3-e470db4b2c85 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:44.429374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9aeb70-e487-4311-a6b3-e470db4b2c85 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:44.441282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1279/2536] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:47.584514 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c3ee8-8772-4754-a12d-272314de6f50 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:47.584959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c3ee8-8772-4754-a12d-272314de6f50 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:47.585111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9c3ee8-8772-4754-a12d-272314de6f50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:47.607662 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-8a9c3ee8-8772-4754-a12d-272314de6f50 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:47.608967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8a9c3ee8-8772-4754-a12d-272314de6f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:47.711950 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1258/2537] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:48:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:47.792556 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9aeb70-e487-4311-a6b3-e470db4b2c85 service nova] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:47.794535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ba9aeb70-e487-4311-a6b3-e470db4b2c85 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:47.887597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9aeb70-e487-4311-a6b3-e470db4b2c85 service nova] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:47.892961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e9118-627e-4d0c-9918-830ccb6b86fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:47.893647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e9118-627e-4d0c-9918-830ccb6b86fe service swift] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:47.894015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e9118-627e-4d0c-9918-830ccb6b86fe service swift] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:47.894354 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849e9118-627e-4d0c-9918-830ccb6b86fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:47.903870 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e9118-627e-4d0c-9918-830ccb6b86fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:47.905588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849e9118-627e-4d0c-9918-830ccb6b86fe service swift] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:47.914515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1280/2538] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:48:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:48:49.494944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c3ee8-8772-4754-a12d-272314de6f50 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:49.497656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8a9c3ee8-8772-4754-a12d-272314de6f50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:49.522095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9c3ee8-8772-4754-a12d-272314de6f50 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:49.538378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089be58c-fcef-446d-9ea9-15c9556738f1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:49.551453 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089be58c-fcef-446d-9ea9-15c9556738f1 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:49.551925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089be58c-fcef-446d-9ea9-15c9556738f1 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:49.552205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089be58c-fcef-446d-9ea9-15c9556738f1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:49.558294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-089be58c-fcef-446d-9ea9-15c9556738f1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:49.576485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089be58c-fcef-446d-9ea9-15c9556738f1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:49.578288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089be58c-fcef-446d-9ea9-15c9556738f1 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:49.609281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1259/2539] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:50.380703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e9118-627e-4d0c-9918-830ccb6b86fe service swift] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:50.468223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849e9118-627e-4d0c-9918-830ccb6b86fe service swift] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:50.473631 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:50.474347 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:50.474827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:50.475105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] PATH_INFO: `/v3/users/0f6ff15ef5a34e1ab53ceb9f613d9071` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:50.477192 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f6ff15ef5a34e1ab53ceb9f613d9071)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:50.479806 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:50.528157 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:50.611310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f6ff15ef5a34e1ab53ceb9f613d9071'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:50.624640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f6ff15ef5a34e1ab53ceb9f613d9071'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:50.625717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f6ff15ef5a34e1ab53ceb9f613d9071'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:50.649020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f6ff15ef5a34e1ab53ceb9f613d9071'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:50.666131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f6ff15ef5a34e1ab53ceb9f613d9071'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:50.678206 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1281/2540] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:48:50 2022] DELETE /identity/v3/users/0f6ff15ef5a34e1ab53ceb9f613d9071 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:50.682714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0ade6-f19b-48d2-86cf-52bab8d5b26f None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:50.683557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0ade6-f19b-48d2-86cf-52bab8d5b26f None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:50.683996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0ade6-f19b-48d2-86cf-52bab8d5b26f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:50.715509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-aea0ade6-f19b-48d2-86cf-52bab8d5b26f None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:50.717013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-aea0ade6-f19b-48d2-86cf-52bab8d5b26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:50.894312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1260/2541] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:50.902293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:50.903767 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:51.014381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901ba6f8-5629-47ef-a098-7f94bf055ac1 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:51.019479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933ba7b-5e26-4c23-88c5-10b58061bbfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:51.020141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ba7b-5e26-4c23-88c5-10b58061bbfc service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:51.020426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ba7b-5e26-4c23-88c5-10b58061bbfc service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:51.020699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933ba7b-5e26-4c23-88c5-10b58061bbfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:51.034279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933ba7b-5e26-4c23-88c5-10b58061bbfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:51.035810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933ba7b-5e26-4c23-88c5-10b58061bbfc service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:51.068716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1282/2542] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:51.576996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0ade6-f19b-48d2-86cf-52bab8d5b26f None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:51.578610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-aea0ade6-f19b-48d2-86cf-52bab8d5b26f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:51.594183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0ade6-f19b-48d2-86cf-52bab8d5b26f None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:51.599163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:51.599789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:51.600063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:51.600491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] PATH_INFO: `/v3/projects/b0ff502834994518adbe6eb1f8876be8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:51.605255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ff502834994518adbe6eb1f8876be8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:51.607177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:51.667268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ff502834994518adbe6eb1f8876be8', 'request_id': 'req-0ffc46c6-4680-4340-96db-680fe15a9692'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:51.673981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ff502834994518adbe6eb1f8876be8', 'request_id': 'req-0ffc46c6-4680-4340-96db-680fe15a9692'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:51.688280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ff502834994518adbe6eb1f8876be8', 'request_id': 'req-0ffc46c6-4680-4340-96db-680fe15a9692'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:51.700798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] The token cache is being invalidate because project b0ff502834994518adbe6eb1f8876be8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:48:51.701098 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:51.701767 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0ffc46c6-4680-4340-96db-680fe15a9692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:48:51.703300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1261/2543] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:48:51 2022] DELETE /identity/v3/projects/b0ff502834994518adbe6eb1f8876be8 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:48:51.708321 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf7e13-384e-4f7b-ab3f-2cc2ebcf628a None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:51.709310 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf7e13-384e-4f7b-ab3f-2cc2ebcf628a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:51.709593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bf7e13-384e-4f7b-ab3f-2cc2ebcf628a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:51.731949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-32bf7e13-384e-4f7b-ab3f-2cc2ebcf628a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:51.733247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-32bf7e13-384e-4f7b-ab3f-2cc2ebcf628a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:51.828802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1283/2544] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:51.914864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc9544-2630-4d61-b482-27e27e973040 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:51.915236 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc9544-2630-4d61-b482-27e27e973040 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:51.915724 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcc9544-2630-4d61-b482-27e27e973040 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:51.937748 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6dcc9544-2630-4d61-b482-27e27e973040 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:51.938958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6dcc9544-2630-4d61-b482-27e27e973040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:51.949711 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1262/2545] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:51.954029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf7e13-384e-4f7b-ab3f-2cc2ebcf628a None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:51.971763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bf7e13-384e-4f7b-ab3f-2cc2ebcf628a None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:51.977775 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2c539-e1e5-4c6a-85cf-a81822175792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:51.982650 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2c539-e1e5-4c6a-85cf-a81822175792 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:51.983191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2c539-e1e5-4c6a-85cf-a81822175792 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:51.983614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a2c539-e1e5-4c6a-85cf-a81822175792 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:51.984989 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2c539-e1e5-4c6a-85cf-a81822175792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:48:51.985646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2c539-e1e5-4c6a-85cf-a81822175792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:51.987284 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a2c539-e1e5-4c6a-85cf-a81822175792 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:52.004079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1284/2546] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:48:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:48:52.008582 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc9544-2630-4d61-b482-27e27e973040 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:52.020514 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcc9544-2630-4d61-b482-27e27e973040 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:52.029387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e834edea-9851-4c65-8e72-a3306faceb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:52.030234 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834edea-9851-4c65-8e72-a3306faceb7a admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:52.030508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834edea-9851-4c65-8e72-a3306faceb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:52.030774 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e834edea-9851-4c65-8e72-a3306faceb7a admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:52.031438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834edea-9851-4c65-8e72-a3306faceb7a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:52.033462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e834edea-9851-4c65-8e72-a3306faceb7a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:52.052355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:52.105023 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1263/2547] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:48:52 2022] POST /identity/v3/projects => generated 373 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:52.106683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2c539-e1e5-4c6a-85cf-a81822175792 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:52.117075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a2c539-e1e5-4c6a-85cf-a81822175792 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:52.149443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:52.152934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b554ca-570b-4a47-9230-6d6307d1c796 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:52.153219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b554ca-570b-4a47-9230-6d6307d1c796 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:52.153497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b554ca-570b-4a47-9230-6d6307d1c796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:52.183363 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c5b554ca-570b-4a47-9230-6d6307d1c796 None None] MFA Rules not processed for user `29018ec15cc147d3919d9047569a4c68`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:52.185245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c5b554ca-570b-4a47-9230-6d6307d1c796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:52.201155 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a8671-3f60-4dfb-94f8-93c6654e34f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:52.202001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a8671-3f60-4dfb-94f8-93c6654e34f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:52.202345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a8671-3f60-4dfb-94f8-93c6654e34f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:52.202686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7a8671-3f60-4dfb-94f8-93c6654e34f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:52.203710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a8671-3f60-4dfb-94f8-93c6654e34f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:52.206619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7a8671-3f60-4dfb-94f8-93c6654e34f6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:52.248793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1285/2548] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:48:52 2022] POST /identity/v3/users => generated 287 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:48:52.250451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a8671-3f60-4dfb-94f8-93c6654e34f6 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:52.261984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7a8671-3f60-4dfb-94f8-93c6654e34f6 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:52.267734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd0957-1e11-454f-b87b-45fb0b53213b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:52.268481 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd0957-1e11-454f-b87b-45fb0b53213b admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:52.268869 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd0957-1e11-454f-b87b-45fb0b53213b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:52.269230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fd0957-1e11-454f-b87b-45fb0b53213b admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:52.270103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd0957-1e11-454f-b87b-45fb0b53213b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:52.271925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fd0957-1e11-454f-b87b-45fb0b53213b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:52.296117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1264/2549] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:52 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:48:52.303993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b554ca-570b-4a47-9230-6d6307d1c796 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:52.304556 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1286/2550] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:52.328643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b554ca-570b-4a47-9230-6d6307d1c796 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:52.338125 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef59d792-6de7-4ff8-8815-963290bf5c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:52.339004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59d792-6de7-4ff8-8815-963290bf5c94 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:52.339463 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59d792-6de7-4ff8-8815-963290bf5c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:52.339899 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef59d792-6de7-4ff8-8815-963290bf5c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:52.341417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59d792-6de7-4ff8-8815-963290bf5c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:52.343190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef59d792-6de7-4ff8-8815-963290bf5c94 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:52.360225 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1265/2551] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:48:52 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:48:52.361157 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd0957-1e11-454f-b87b-45fb0b53213b admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:52.383037 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fd0957-1e11-454f-b87b-45fb0b53213b admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:52.392824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797629f-1388-458a-b0a5-5133537bd149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:52.393648 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797629f-1388-458a-b0a5-5133537bd149 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:52.397660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797629f-1388-458a-b0a5-5133537bd149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:52.398070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b797629f-1388-458a-b0a5-5133537bd149 admin admin] PATH_INFO: `/v3/projects/9b3e57c72cec4c128a09749da5a90a5d/users/e5593dc049144447859a48e458717295/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:52.436779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797629f-1388-458a-b0a5-5133537bd149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b3e57c72cec4c128a09749da5a90a5d, user_id=e5593dc049144447859a48e458717295, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:52.439874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b797629f-1388-458a-b0a5-5133537bd149 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:52.465826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1287/2552] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:48:52 2022] PUT /identity/v3/projects/9b3e57c72cec4c128a09749da5a90a5d/users/e5593dc049144447859a48e458717295/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:48:52.471038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e78d3-4294-495e-b242-487f140e9340 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:52.471515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e78d3-4294-495e-b242-487f140e9340 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:52.471863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5e78d3-4294-495e-b242-487f140e9340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:52.524699 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1b5e78d3-4294-495e-b242-487f140e9340 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:52.528483 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1b5e78d3-4294-495e-b242-487f140e9340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:52.617750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1266/2553] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:52.629674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797629f-1388-458a-b0a5-5133537bd149 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:52.631584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b797629f-1388-458a-b0a5-5133537bd149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:52.739366 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b797629f-1388-458a-b0a5-5133537bd149 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:52.745893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ac6f9-2e09-4ebc-9d27-43d518cea1ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:52.746758 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ac6f9-2e09-4ebc-9d27-43d518cea1ec service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:52.747034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ac6f9-2e09-4ebc-9d27-43d518cea1ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:52.747304 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91ac6f9-2e09-4ebc-9d27-43d518cea1ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:52.759044 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ac6f9-2e09-4ebc-9d27-43d518cea1ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:52.760945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91ac6f9-2e09-4ebc-9d27-43d518cea1ec service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:52.781950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1288/2554] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:52.939120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e78d3-4294-495e-b242-487f140e9340 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:52.940751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1b5e78d3-4294-495e-b242-487f140e9340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:52.959530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5e78d3-4294-495e-b242-487f140e9340 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:52.965493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2b782-a2f0-45ad-abaa-1cf657089763 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:52.966424 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2b782-a2f0-45ad-abaa-1cf657089763 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:52.967351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2b782-a2f0-45ad-abaa-1cf657089763 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:52.968121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2b782-a2f0-45ad-abaa-1cf657089763 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:52.971181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-7bd2b782-a2f0-45ad-abaa-1cf657089763 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:52.989737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2b782-a2f0-45ad-abaa-1cf657089763 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:52.991660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd2b782-a2f0-45ad-abaa-1cf657089763 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:53.009310 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1267/2555] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:53.251211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ac6f9-2e09-4ebc-9d27-43d518cea1ec service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:53.274956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91ac6f9-2e09-4ebc-9d27-43d518cea1ec service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:53.287568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508be53-366b-47c7-ac61-d48a687d4244 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:53.288338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508be53-366b-47c7-ac61-d48a687d4244 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:53.291697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508be53-366b-47c7-ac61-d48a687d4244 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:53.292345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7508be53-366b-47c7-ac61-d48a687d4244 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:53.295071 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7508be53-366b-47c7-ac61-d48a687d4244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:53.318177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508be53-366b-47c7-ac61-d48a687d4244 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:53.322934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7508be53-366b-47c7-ac61-d48a687d4244 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:53.341642 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1289/2556] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:53.694905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a76918-8b1c-4c19-bf4a-958f8279ed02 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:53.695288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a76918-8b1c-4c19-bf4a-958f8279ed02 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:53.695684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a76918-8b1c-4c19-bf4a-958f8279ed02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:53.763324 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-56a76918-8b1c-4c19-bf4a-958f8279ed02 None None] MFA Rules not processed for user `e5593dc049144447859a48e458717295`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:53.765149 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-56a76918-8b1c-4c19-bf4a-958f8279ed02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:53.877913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:48:53.911550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508be53-366b-47c7-ac61-d48a687d4244 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:53.912406 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1268/2557] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:53.914246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7508be53-366b-47c7-ac61-d48a687d4244 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:54.060164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7508be53-366b-47c7-ac61-d48a687d4244 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:54.066664 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c3e321-0ca8-4ba1-81e2-e290f980cd8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:54.067884 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3e321-0ca8-4ba1-81e2-e290f980cd8b service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:54.068223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3e321-0ca8-4ba1-81e2-e290f980cd8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:54.068580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c3e321-0ca8-4ba1-81e2-e290f980cd8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:54.098005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3e321-0ca8-4ba1-81e2-e290f980cd8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:54.104950 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c3e321-0ca8-4ba1-81e2-e290f980cd8b service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:54.127988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1290/2558] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:54.216419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a76918-8b1c-4c19-bf4a-958f8279ed02 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:54.218415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-56a76918-8b1c-4c19-bf4a-958f8279ed02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:54.336612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a76918-8b1c-4c19-bf4a-958f8279ed02 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:54.345777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a12c49-558e-4418-82e3-8d68cd2ad956 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:54.347602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a12c49-558e-4418-82e3-8d68cd2ad956 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:54.348906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a12c49-558e-4418-82e3-8d68cd2ad956 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:54.349412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a12c49-558e-4418-82e3-8d68cd2ad956 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:54.351753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-69a12c49-558e-4418-82e3-8d68cd2ad956 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:54.366718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a12c49-558e-4418-82e3-8d68cd2ad956 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:54.369364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a12c49-558e-4418-82e3-8d68cd2ad956 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:54.390096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1269/2559] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:48:55.843578 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d250f-43c3-4b30-9d08-e244fc59fd44 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:55.844857 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d250f-43c3-4b30-9d08-e244fc59fd44 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:55.845304 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033d250f-43c3-4b30-9d08-e244fc59fd44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:55.930679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-033d250f-43c3-4b30-9d08-e244fc59fd44 None None] MFA Rules not processed for user `e5593dc049144447859a48e458717295`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:48:55.932931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-033d250f-43c3-4b30-9d08-e244fc59fd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:55.952429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1291/2560] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:48:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:48:56.562829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a12c49-558e-4418-82e3-8d68cd2ad956 service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:48:56.582916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a12c49-558e-4418-82e3-8d68cd2ad956 service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:48:56.589435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd241172-bf10-4524-87dd-6137db9d798b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:48:56.591219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd241172-bf10-4524-87dd-6137db9d798b service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:48:56.591733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd241172-bf10-4524-87dd-6137db9d798b service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:48:56.592272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd241172-bf10-4524-87dd-6137db9d798b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:48:56.595684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-fd241172-bf10-4524-87dd-6137db9d798b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:48:56.612583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd241172-bf10-4524-87dd-6137db9d798b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:48:56.614528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd241172-bf10-4524-87dd-6137db9d798b service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:48:56.629611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1270/2561] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:48:56 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:01.590634 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d250f-43c3-4b30-9d08-e244fc59fd44 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:01.593116 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-033d250f-43c3-4b30-9d08-e244fc59fd44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:01.602046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:01.727186 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033d250f-43c3-4b30-9d08-e244fc59fd44 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:01.735715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a255d9b-bf85-436d-a218-21a691e3f237 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:01.736738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a255d9b-bf85-436d-a218-21a691e3f237 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:01.737395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a255d9b-bf85-436d-a218-21a691e3f237 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:01.738112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a255d9b-bf85-436d-a218-21a691e3f237 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:01.740441 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-4a255d9b-bf85-436d-a218-21a691e3f237 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:01.761210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a255d9b-bf85-436d-a218-21a691e3f237 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:01.763249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a255d9b-bf85-436d-a218-21a691e3f237 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:01.778331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1292/2562] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:06.018769 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd241172-bf10-4524-87dd-6137db9d798b service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:06.028506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd241172-bf10-4524-87dd-6137db9d798b service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:06.032358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cfb332-e682-4469-bba5-9001e16f51c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:06.033031 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb332-e682-4469-bba5-9001e16f51c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:06.033306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb332-e682-4469-bba5-9001e16f51c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:06.033575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cfb332-e682-4469-bba5-9001e16f51c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:06.035375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b7cfb332-e682-4469-bba5-9001e16f51c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:06.048170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfb332-e682-4469-bba5-9001e16f51c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:06.049714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cfb332-e682-4469-bba5-9001e16f51c1 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:06.059714 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1271/2563] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:49:09.002335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091cdcf-887a-4949-9791-b8ca2205e4bc None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:09.003221 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091cdcf-887a-4949-9791-b8ca2205e4bc None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:09.003791 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c091cdcf-887a-4949-9791-b8ca2205e4bc None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:09.005561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1293/2564] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:49:09.013552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535252e3-4f92-460c-9d28-771a71b70c8a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:09.014128 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535252e3-4f92-460c-9d28-771a71b70c8a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:09.014478 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535252e3-4f92-460c-9d28-771a71b70c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:09.052121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-535252e3-4f92-460c-9d28-771a71b70c8a None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:09.053599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-535252e3-4f92-460c-9d28-771a71b70c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:09.077787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1272/2565] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:49:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:49:09.091530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091cdcf-887a-4949-9791-b8ca2205e4bc None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:09.095171 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c091cdcf-887a-4949-9791-b8ca2205e4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:09.199887 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c091cdcf-887a-4949-9791-b8ca2205e4bc None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:09.204970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416ae32c-117f-4bf2-8fdf-dcd3e00bb9ee service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42cf265450454ff9a27a29093336205e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:09.205832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae32c-117f-4bf2-8fdf-dcd3e00bb9ee service glance] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:09.206286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae32c-117f-4bf2-8fdf-dcd3e00bb9ee service glance] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:09.207025 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ae32c-117f-4bf2-8fdf-dcd3e00bb9ee service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:09.223089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ae32c-117f-4bf2-8fdf-dcd3e00bb9ee service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:09.225905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416ae32c-117f-4bf2-8fdf-dcd3e00bb9ee service glance] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:09.243973 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1294/2566] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:09.358888 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c955446-ff00-498b-a840-188b4ddae947 None None] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:09.359328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c955446-ff00-498b-a840-188b4ddae947 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:09.359720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c955446-ff00-498b-a840-188b4ddae947 None None] PATH_INFO: `/` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:09.361325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1273/2567] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:49:09.371975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03e5b7-9d50-47a9-8bd7-a2b687226c3a None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:09.372626 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03e5b7-9d50-47a9-8bd7-a2b687226c3a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:09.373128 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd03e5b7-9d50-47a9-8bd7-a2b687226c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:09.394547 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-dd03e5b7-9d50-47a9-8bd7-a2b687226c3a None None] MFA Rules not processed for user `2658435f74a947f39e207465ebff22d8`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:09.396202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-dd03e5b7-9d50-47a9-8bd7-a2b687226c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:09.405490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1295/2568] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:49:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:09.427455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c955446-ff00-498b-a840-188b4ddae947 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:09.444450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c955446-ff00-498b-a840-188b4ddae947 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:09.448797 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dc60b3-6f3e-4f28-8933-8c6e47d4a72d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:09.449665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc60b3-6f3e-4f28-8933-8c6e47d4a72d service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:09.450058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc60b3-6f3e-4f28-8933-8c6e47d4a72d service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:09.450422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dc60b3-6f3e-4f28-8933-8c6e47d4a72d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:09.461344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc60b3-6f3e-4f28-8933-8c6e47d4a72d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:09.463245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dc60b3-6f3e-4f28-8933-8c6e47d4a72d service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:09.477275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1274/2569] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:09 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:09.640002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26ff34-8796-4bff-a887-b10ae686c268 None None] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:09.640280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26ff34-8796-4bff-a887-b10ae686c268 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:09.640550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26ff34-8796-4bff-a887-b10ae686c268 None None] PATH_INFO: `/` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:09.642172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1296/2570] 10.222.0.39 () {54 vars in 921 bytes} [Tue Sep 6 22:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 06 22:49:09.648582 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced8874-272d-48e4-8342-b1e95e0ae4bc None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:09.649002 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced8874-272d-48e4-8342-b1e95e0ae4bc None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:09.649346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ced8874-272d-48e4-8342-b1e95e0ae4bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:09.672856 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4ced8874-272d-48e4-8342-b1e95e0ae4bc None None] MFA Rules not processed for user `b16947313dad4d59a97cc8b016ee54c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:09.674426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4ced8874-272d-48e4-8342-b1e95e0ae4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:09.688192 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1275/2571] 10.222.0.39 () {58 vars in 1018 bytes} [Tue Sep 6 22:49:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:49:09.712409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26ff34-8796-4bff-a887-b10ae686c268 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:09.743743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26ff34-8796-4bff-a887-b10ae686c268 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:09.750921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fae774-31a4-4d92-a5b8-73b6e2ea0b08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:09.751544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae774-31a4-4d92-a5b8-73b6e2ea0b08 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:09.751820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae774-31a4-4d92-a5b8-73b6e2ea0b08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:09.752091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fae774-31a4-4d92-a5b8-73b6e2ea0b08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:09.766110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fae774-31a4-4d92-a5b8-73b6e2ea0b08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:09.768783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fae774-31a4-4d92-a5b8-73b6e2ea0b08 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:09.783709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1297/2572] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:12.240413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced8874-272d-48e4-8342-b1e95e0ae4bc None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:12.242610 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4ced8874-272d-48e4-8342-b1e95e0ae4bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:12.265792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ced8874-272d-48e4-8342-b1e95e0ae4bc None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:12.273281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e1af34-9b3c-422a-a4bc-0818d4c7ee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:12.278674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1af34-9b3c-422a-a4bc-0818d4c7ee10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:12.278674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1af34-9b3c-422a-a4bc-0818d4c7ee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:12.278674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e1af34-9b3c-422a-a4bc-0818d4c7ee10 admin admin] PATH_INFO: `/v3/users/4dd9f0bff3044d7db561dff21ac0e7ff` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:12.279675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e1af34-9b3c-422a-a4bc-0818d4c7ee10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4dd9f0bff3044d7db561dff21ac0e7ff)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:12.282048 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e1af34-9b3c-422a-a4bc-0818d4c7ee10 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:12.344446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c5e1af34-9b3c-422a-a4bc-0818d4c7ee10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4dd9f0bff3044d7db561dff21ac0e7ff'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:12.345152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c5e1af34-9b3c-422a-a4bc-0818d4c7ee10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4dd9f0bff3044d7db561dff21ac0e7ff'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:12.352954 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c5e1af34-9b3c-422a-a4bc-0818d4c7ee10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4dd9f0bff3044d7db561dff21ac0e7ff'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:12.361209 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c5e1af34-9b3c-422a-a4bc-0818d4c7ee10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4dd9f0bff3044d7db561dff21ac0e7ff'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:12.373935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c5e1af34-9b3c-422a-a4bc-0818d4c7ee10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4dd9f0bff3044d7db561dff21ac0e7ff'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:12.380000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1276/2573] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:12 2022] DELETE /identity/v3/users/4dd9f0bff3044d7db561dff21ac0e7ff => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:12.633785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fae774-31a4-4d92-a5b8-73b6e2ea0b08 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:12.649818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fae774-31a4-4d92-a5b8-73b6e2ea0b08 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:12.727533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:12.729552 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:12.731788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:12.732086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] PATH_INFO: `/v3/projects/817010ab38864d8aafb61140eb176da1` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:12.734465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] RBAC: Authorizing `identity:delete_project(project_id=817010ab38864d8aafb61140eb176da1)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:12.736725 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:12.754056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:12.786699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '817010ab38864d8aafb61140eb176da1', 'request_id': 'req-6024ff44-3b8f-491e-aa05-fe492a19615a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:12.795722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '817010ab38864d8aafb61140eb176da1', 'request_id': 'req-6024ff44-3b8f-491e-aa05-fe492a19615a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:12.802113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '817010ab38864d8aafb61140eb176da1', 'request_id': 'req-6024ff44-3b8f-491e-aa05-fe492a19615a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:12.813844 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] The token cache is being invalidate because project 817010ab38864d8aafb61140eb176da1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:12.814133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:12.814774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6024ff44-3b8f-491e-aa05-fe492a19615a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:12.816322 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1298/2574] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:12 2022] DELETE /identity/v3/projects/817010ab38864d8aafb61140eb176da1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:12.821195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd02d-5e29-4111-ac6f-f8010686c3f1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:12.821476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd02d-5e29-4111-ac6f-f8010686c3f1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:12.821800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768fd02d-5e29-4111-ac6f-f8010686c3f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:12.836820 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:12.845130 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-768fd02d-5e29-4111-ac6f-f8010686c3f1 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:12.846982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-768fd02d-5e29-4111-ac6f-f8010686c3f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:12.906038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1277/2575] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:12.913877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118232ae-ca15-4373-b8a1-c7ebbfb4d038 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:12.914165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118232ae-ca15-4373-b8a1-c7ebbfb4d038 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:12.914439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-118232ae-ca15-4373-b8a1-c7ebbfb4d038 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:12.934852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-118232ae-ca15-4373-b8a1-c7ebbfb4d038 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:12.936130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-118232ae-ca15-4373-b8a1-c7ebbfb4d038 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:12.947404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1299/2576] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:12.952733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fd02d-5e29-4111-ac6f-f8010686c3f1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:12.966564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768fd02d-5e29-4111-ac6f-f8010686c3f1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:12.974320 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38124fa7-b7f2-43e4-9745-a55caf77724f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:12.975013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38124fa7-b7f2-43e4-9745-a55caf77724f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:12.975305 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38124fa7-b7f2-43e4-9745-a55caf77724f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:12.975553 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38124fa7-b7f2-43e4-9745-a55caf77724f admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:12.976251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38124fa7-b7f2-43e4-9745-a55caf77724f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:49:12.976572 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38124fa7-b7f2-43e4-9745-a55caf77724f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:12.978397 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38124fa7-b7f2-43e4-9745-a55caf77724f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:12.992889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1278/2577] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:49:12.997572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118232ae-ca15-4373-b8a1-c7ebbfb4d038 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:13.007047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-118232ae-ca15-4373-b8a1-c7ebbfb4d038 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:13.012540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c787dcb-1e94-41b2-81d2-6c61fa4cd38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:13.013164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c787dcb-1e94-41b2-81d2-6c61fa4cd38b admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:13.013443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c787dcb-1e94-41b2-81d2-6c61fa4cd38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:13.013712 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c787dcb-1e94-41b2-81d2-6c61fa4cd38b admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:13.014349 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c787dcb-1e94-41b2-81d2-6c61fa4cd38b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:13.016224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c787dcb-1e94-41b2-81d2-6c61fa4cd38b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:13.046153 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1300/2578] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:12 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:13.049909 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38124fa7-b7f2-43e4-9745-a55caf77724f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:13.062284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38124fa7-b7f2-43e4-9745-a55caf77724f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:13.132660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc041d89-f0b2-431b-b998-68fa5d0ab1d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:13.133300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc041d89-f0b2-431b-b998-68fa5d0ab1d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:13.133581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc041d89-f0b2-431b-b998-68fa5d0ab1d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:13.133881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc041d89-f0b2-431b-b998-68fa5d0ab1d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:13.135461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc041d89-f0b2-431b-b998-68fa5d0ab1d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:13.138003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc041d89-f0b2-431b-b998-68fa5d0ab1d0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:13.142239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e47d36-f1a9-4d8b-ae3f-5f3ab2761302 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:13.142871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e47d36-f1a9-4d8b-ae3f-5f3ab2761302 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:13.143201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e47d36-f1a9-4d8b-ae3f-5f3ab2761302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:13.161830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1279/2579] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:13 2022] POST /identity/v3/users => generated 286 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:13.165423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-23e47d36-f1a9-4d8b-ae3f-5f3ab2761302 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:13.165822 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc041d89-f0b2-431b-b998-68fa5d0ab1d0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:13.166661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-23e47d36-f1a9-4d8b-ae3f-5f3ab2761302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:13.175742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc041d89-f0b2-431b-b998-68fa5d0ab1d0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:13.181131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fc8bd-a9b7-43bd-ba8e-e856d493f459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:13.181761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fc8bd-a9b7-43bd-ba8e-e856d493f459 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:13.182849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fc8bd-a9b7-43bd-ba8e-e856d493f459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:13.182849 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14fc8bd-a9b7-43bd-ba8e-e856d493f459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:13.183544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fc8bd-a9b7-43bd-ba8e-e856d493f459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:13.185251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14fc8bd-a9b7-43bd-ba8e-e856d493f459 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:13.196909 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1280/2580] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:49:13.201859 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fc8bd-a9b7-43bd-ba8e-e856d493f459 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:13.211728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14fc8bd-a9b7-43bd-ba8e-e856d493f459 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:13.216653 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547d6d0-6f68-4bb3-8e37-8fd303085e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:13.217349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547d6d0-6f68-4bb3-8e37-8fd303085e5a admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:13.217691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547d6d0-6f68-4bb3-8e37-8fd303085e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:13.217965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2547d6d0-6f68-4bb3-8e37-8fd303085e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:13.218599 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547d6d0-6f68-4bb3-8e37-8fd303085e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:13.220241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2547d6d0-6f68-4bb3-8e37-8fd303085e5a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:13.230493 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1301/2581] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:49:13 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:49:13.230797 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1281/2582] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:49:13.234697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e47d36-f1a9-4d8b-ae3f-5f3ab2761302 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:13.246481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547d6d0-6f68-4bb3-8e37-8fd303085e5a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:13.248115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e47d36-f1a9-4d8b-ae3f-5f3ab2761302 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:13.248627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-2547d6d0-6f68-4bb3-8e37-8fd303085e5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:13.254811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1dfda2-76cf-49e8-8f70-7d1a275371f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:13.255665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dfda2-76cf-49e8-8f70-7d1a275371f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:13.255982 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dfda2-76cf-49e8-8f70-7d1a275371f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:13.256209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1dfda2-76cf-49e8-8f70-7d1a275371f1 admin admin] PATH_INFO: `/v3/projects/387afebc94bd43888a281e078e939b43/users/5bd22845b8094127880995c1ef4e720e/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:13.274243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dfda2-76cf-49e8-8f70-7d1a275371f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=387afebc94bd43888a281e078e939b43, user_id=5bd22845b8094127880995c1ef4e720e, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:13.277444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1dfda2-76cf-49e8-8f70-7d1a275371f1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:13.293785 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1302/2583] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:13 2022] PUT /identity/v3/projects/387afebc94bd43888a281e078e939b43/users/5bd22845b8094127880995c1ef4e720e/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:13.305052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf271cc8-7d10-4148-83f8-f14747c23725 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:13.305473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf271cc8-7d10-4148-83f8-f14747c23725 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:13.306005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf271cc8-7d10-4148-83f8-f14747c23725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:13.336521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-bf271cc8-7d10-4148-83f8-f14747c23725 None None] MFA Rules not processed for user `5bd22845b8094127880995c1ef4e720e`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:13.337840 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-bf271cc8-7d10-4148-83f8-f14747c23725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:13.343800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:13.350556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2547d6d0-6f68-4bb3-8e37-8fd303085e5a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:13.388962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:13.413553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1303/2584] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:13 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:49:13.424064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf271cc8-7d10-4148-83f8-f14747c23725 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:13.425765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-bf271cc8-7d10-4148-83f8-f14747c23725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:13.428801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907594d2-8bd8-446b-82ed-fe7951c1ece3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:13.429318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907594d2-8bd8-446b-82ed-fe7951c1ece3 service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:13.429614 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907594d2-8bd8-446b-82ed-fe7951c1ece3 service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:13.429884 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907594d2-8bd8-446b-82ed-fe7951c1ece3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:13.510871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf271cc8-7d10-4148-83f8-f14747c23725 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:13.516660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056b92b-97ae-4f52-8dbf-d76b2db3c8b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:13.517299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056b92b-97ae-4f52-8dbf-d76b2db3c8b5 service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:13.517568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056b92b-97ae-4f52-8dbf-d76b2db3c8b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:13.517835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0056b92b-97ae-4f52-8dbf-d76b2db3c8b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:13.520268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907594d2-8bd8-446b-82ed-fe7951c1ece3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:13.521960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907594d2-8bd8-446b-82ed-fe7951c1ece3 service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:13.525478 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056b92b-97ae-4f52-8dbf-d76b2db3c8b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:13.527822 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0056b92b-97ae-4f52-8dbf-d76b2db3c8b5 service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:13.531460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1282/2585] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:49:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:49:13.532671 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55448fb-13ac-460b-b5f8-45081bc66191 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:13.533223 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55448fb-13ac-460b-b5f8-45081bc66191 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:13.533622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55448fb-13ac-460b-b5f8-45081bc66191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:13.537564 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1304/2586] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:49:13.556106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a55448fb-13ac-460b-b5f8-45081bc66191 None None] MFA Rules not processed for user `e5593dc049144447859a48e458717295`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:13.558035 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a55448fb-13ac-460b-b5f8-45081bc66191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:13.664801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1283/2587] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:16.498335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056b92b-97ae-4f52-8dbf-d76b2db3c8b5 service nova] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:16.585381 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0056b92b-97ae-4f52-8dbf-d76b2db3c8b5 service nova] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:16.591003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:16.591724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:16.592001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:16.592340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] PATH_INFO: `/v3/users/5bd22845b8094127880995c1ef4e720e` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:16.593518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd22845b8094127880995c1ef4e720e)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:16.595458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:16.649748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd22845b8094127880995c1ef4e720e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:16.658741 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd22845b8094127880995c1ef4e720e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:16.659603 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd22845b8094127880995c1ef4e720e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:16.673099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd22845b8094127880995c1ef4e720e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:16.675014 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:16.683994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd22845b8094127880995c1ef4e720e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:16.694160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1305/2588] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:16 2022] DELETE /identity/v3/users/5bd22845b8094127880995c1ef4e720e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:16.695885 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0e383-5caa-4515-9972-daf47af9e1b4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:16.696361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0e383-5caa-4515-9972-daf47af9e1b4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:16.696547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a0e383-5caa-4515-9972-daf47af9e1b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:16.720486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-37a0e383-5caa-4515-9972-daf47af9e1b4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:16.721732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-37a0e383-5caa-4515-9972-daf47af9e1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:16.798416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:16.814426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1284/2589] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:16.829632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:16.831502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:16.923389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38260d5-0af9-473d-b98c-60411b326b7e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:16.928004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3101da76-ea98-4049-8a54-51a0a6ef8c2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:16.928919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101da76-ea98-4049-8a54-51a0a6ef8c2b service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:16.929559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101da76-ea98-4049-8a54-51a0a6ef8c2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:16.930017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3101da76-ea98-4049-8a54-51a0a6ef8c2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:16.941011 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101da76-ea98-4049-8a54-51a0a6ef8c2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:16.942618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3101da76-ea98-4049-8a54-51a0a6ef8c2b service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:16.954456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1306/2590] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:17.297476 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0e383-5caa-4515-9972-daf47af9e1b4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:17.299516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-37a0e383-5caa-4515-9972-daf47af9e1b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:17.359104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a0e383-5caa-4515-9972-daf47af9e1b4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:17.371500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:17.372187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:17.372440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:17.372717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] PATH_INFO: `/v3/projects/387afebc94bd43888a281e078e939b43` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:17.383147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=387afebc94bd43888a281e078e939b43)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:17.386003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:17.474239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '387afebc94bd43888a281e078e939b43', 'request_id': 'req-18c2da61-b341-445b-a551-70671b443e39'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:17.483807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '387afebc94bd43888a281e078e939b43', 'request_id': 'req-18c2da61-b341-445b-a551-70671b443e39'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:17.497471 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '387afebc94bd43888a281e078e939b43', 'request_id': 'req-18c2da61-b341-445b-a551-70671b443e39'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:17.534211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] The token cache is being invalidate because project 387afebc94bd43888a281e078e939b43 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:17.534811 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:17.535530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-18c2da61-b341-445b-a551-70671b443e39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:17.537116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1285/2591] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:17 2022] DELETE /identity/v3/projects/387afebc94bd43888a281e078e939b43 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:17.542902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feecdbf-9b4c-4683-8d55-c71b277240a6 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:17.543409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feecdbf-9b4c-4683-8d55-c71b277240a6 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:17.543512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2feecdbf-9b4c-4683-8d55-c71b277240a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:17.576704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2feecdbf-9b4c-4683-8d55-c71b277240a6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:17.576704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2feecdbf-9b4c-4683-8d55-c71b277240a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:17.664672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1307/2592] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:17.672727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617b86c-1438-4447-847d-77b5b0624573 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:17.673043 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617b86c-1438-4447-847d-77b5b0624573 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:17.673317 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a617b86c-1438-4447-847d-77b5b0624573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:17.703290 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a617b86c-1438-4447-847d-77b5b0624573 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:17.705350 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a617b86c-1438-4447-847d-77b5b0624573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:17.714263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1286/2593] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:17.718518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feecdbf-9b4c-4683-8d55-c71b277240a6 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:17.730661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2feecdbf-9b4c-4683-8d55-c71b277240a6 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:17.748925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37dbf0-2602-4647-a4cd-3bd4bcaddc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:17.749718 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37dbf0-2602-4647-a4cd-3bd4bcaddc3a admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:17.750020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37dbf0-2602-4647-a4cd-3bd4bcaddc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:17.750361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37dbf0-2602-4647-a4cd-3bd4bcaddc3a admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:17.751123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37dbf0-2602-4647-a4cd-3bd4bcaddc3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:49:17.751427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37dbf0-2602-4647-a4cd-3bd4bcaddc3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:17.753515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37dbf0-2602-4647-a4cd-3bd4bcaddc3a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:17.778339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1308/2594] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:49:17.783267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617b86c-1438-4447-847d-77b5b0624573 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:17.793225 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a617b86c-1438-4447-847d-77b5b0624573 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:17.798186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb45bb5-8029-4f9d-adcb-3980b3a509b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:17.799369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb45bb5-8029-4f9d-adcb-3980b3a509b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:17.799614 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb45bb5-8029-4f9d-adcb-3980b3a509b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:17.799993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb45bb5-8029-4f9d-adcb-3980b3a509b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:17.801965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb45bb5-8029-4f9d-adcb-3980b3a509b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:17.804341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb45bb5-8029-4f9d-adcb-3980b3a509b2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:17.848463 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1287/2595] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:17 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:17.849675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37dbf0-2602-4647-a4cd-3bd4bcaddc3a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:17.861146 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37dbf0-2602-4647-a4cd-3bd4bcaddc3a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:17.921895 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b606c-5415-4beb-b070-f72af801c4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:17.922511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b606c-5415-4beb-b070-f72af801c4ac admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:17.922788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b606c-5415-4beb-b070-f72af801c4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:17.923058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b606c-5415-4beb-b070-f72af801c4ac admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:17.923684 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b606c-5415-4beb-b070-f72af801c4ac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:17.925878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b606c-5415-4beb-b070-f72af801c4ac admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:17.952699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1309/2596] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:17 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:17.953971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb45bb5-8029-4f9d-adcb-3980b3a509b2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:17.965178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb45bb5-8029-4f9d-adcb-3980b3a509b2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:17.969839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664e591-6e18-4518-bfcd-1bd031b2d5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:17.970510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664e591-6e18-4518-bfcd-1bd031b2d5fe admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:17.970785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664e591-6e18-4518-bfcd-1bd031b2d5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:17.971051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c664e591-6e18-4518-bfcd-1bd031b2d5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:17.971678 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664e591-6e18-4518-bfcd-1bd031b2d5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:17.973291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c664e591-6e18-4518-bfcd-1bd031b2d5fe admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:17.993850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1288/2597] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:17.998008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b606c-5415-4beb-b070-f72af801c4ac admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:18.009809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b606c-5415-4beb-b070-f72af801c4ac admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:18.016180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6741c6db-0e17-41c0-a5d2-f65b62d04af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:18.017062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741c6db-0e17-41c0-a5d2-f65b62d04af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:18.017552 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741c6db-0e17-41c0-a5d2-f65b62d04af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:18.017714 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6741c6db-0e17-41c0-a5d2-f65b62d04af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:18.018768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741c6db-0e17-41c0-a5d2-f65b62d04af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:18.020774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6741c6db-0e17-41c0-a5d2-f65b62d04af6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:18.027104 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:18.044518 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664e591-6e18-4518-bfcd-1bd031b2d5fe admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:18.052076 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1310/2598] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:17 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:18.056340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c664e591-6e18-4518-bfcd-1bd031b2d5fe admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:18.062735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e64b24-7fd3-4e4e-b9e4-2c5f45ed3217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:18.063369 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e64b24-7fd3-4e4e-b9e4-2c5f45ed3217 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:18.063698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e64b24-7fd3-4e4e-b9e4-2c5f45ed3217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:18.064046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e64b24-7fd3-4e4e-b9e4-2c5f45ed3217 admin admin] PATH_INFO: `/v3/projects/480bbdf711b5423ea40389a20dff282c/users/6572838491454d84ba3ba516a6af82c9/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:18.091666 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e64b24-7fd3-4e4e-b9e4-2c5f45ed3217 admin admin] RBAC: Authorizing `identity:create_grant(project_id=480bbdf711b5423ea40389a20dff282c, user_id=6572838491454d84ba3ba516a6af82c9, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:18.094245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e64b24-7fd3-4e4e-b9e4-2c5f45ed3217 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:18.111058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1289/2599] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:18 2022] PUT /identity/v3/projects/480bbdf711b5423ea40389a20dff282c/users/6572838491454d84ba3ba516a6af82c9/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:49:18.122533 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db0343-37d7-45cd-ab24-aaff98b9823c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:18.122841 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db0343-37d7-45cd-ab24-aaff98b9823c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:18.123309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0db0343-37d7-45cd-ab24-aaff98b9823c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:18.160516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-f0db0343-37d7-45cd-ab24-aaff98b9823c None None] MFA Rules not processed for user `6572838491454d84ba3ba516a6af82c9`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:18.161905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f0db0343-37d7-45cd-ab24-aaff98b9823c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:18.233469 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1311/2600] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:18.244551 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e64b24-7fd3-4e4e-b9e4-2c5f45ed3217 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:18.247019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-69e64b24-7fd3-4e4e-b9e4-2c5f45ed3217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:18.326673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e64b24-7fd3-4e4e-b9e4-2c5f45ed3217 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:18.332641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0942da-cb86-403e-a306-44e84d5602e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:18.333291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0942da-cb86-403e-a306-44e84d5602e1 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:18.333571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0942da-cb86-403e-a306-44e84d5602e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:18.333844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0942da-cb86-403e-a306-44e84d5602e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:18.340294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:18.343190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0942da-cb86-403e-a306-44e84d5602e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:18.345109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0942da-cb86-403e-a306-44e84d5602e1 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:18.355987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1290/2601] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:18.746176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db0343-37d7-45cd-ab24-aaff98b9823c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:18.852935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0db0343-37d7-45cd-ab24-aaff98b9823c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:18.859142 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:18.859834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:18.860105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:18.860378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] PATH_INFO: `/v3/users/6572838491454d84ba3ba516a6af82c9` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:18.861581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6572838491454d84ba3ba516a6af82c9)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:18.863574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:18.957678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6572838491454d84ba3ba516a6af82c9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:19.010097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6572838491454d84ba3ba516a6af82c9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:19.011271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6572838491454d84ba3ba516a6af82c9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:19.052793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6572838491454d84ba3ba516a6af82c9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:19.060855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6572838491454d84ba3ba516a6af82c9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:19.068223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1312/2602] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:18 2022] DELETE /identity/v3/users/6572838491454d84ba3ba516a6af82c9 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:19.073816 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4193b-5fdc-405c-89b1-5df994c996c4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:19.077629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4193b-5fdc-405c-89b1-5df994c996c4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:19.077629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf4193b-5fdc-405c-89b1-5df994c996c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:19.125847 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4bf4193b-5fdc-405c-89b1-5df994c996c4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:19.127235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4bf4193b-5fdc-405c-89b1-5df994c996c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:19.217982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1291/2603] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:19.233220 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:19.235090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:19.252094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:19.327784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc236a94-3690-4b0e-9833-97946965149c admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:19.347546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf4546-bc60-4ba3-b8a0-f75484e7fa6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:19.348616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf4546-bc60-4ba3-b8a0-f75484e7fa6b service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:19.348934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf4546-bc60-4ba3-b8a0-f75484e7fa6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:19.349210 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf4546-bc60-4ba3-b8a0-f75484e7fa6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:19.365245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf4546-bc60-4ba3-b8a0-f75484e7fa6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:19.368125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf4546-bc60-4ba3-b8a0-f75484e7fa6b service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:19.387534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1313/2604] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:19.766343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4193b-5fdc-405c-89b1-5df994c996c4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:19.768085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4bf4193b-5fdc-405c-89b1-5df994c996c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:19.783353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf4193b-5fdc-405c-89b1-5df994c996c4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:19.804673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:19.805302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:19.806071 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:19.806399 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] PATH_INFO: `/v3/projects/480bbdf711b5423ea40389a20dff282c` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:19.809673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=480bbdf711b5423ea40389a20dff282c)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:19.828427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:19.905455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '480bbdf711b5423ea40389a20dff282c', 'request_id': 'req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:19.914987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '480bbdf711b5423ea40389a20dff282c', 'request_id': 'req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:19.929186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '480bbdf711b5423ea40389a20dff282c', 'request_id': 'req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:19.948184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] The token cache is being invalidate because project 480bbdf711b5423ea40389a20dff282c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:19.948861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:19.949740 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-eb835b80-c9cc-4b33-8f85-0a9f80a064bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:19.956565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1292/2605] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:19 2022] DELETE /identity/v3/projects/480bbdf711b5423ea40389a20dff282c => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:19.957824 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9c451-1ba4-4b5e-b799-d7f99df29875 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:19.958521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9c451-1ba4-4b5e-b799-d7f99df29875 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:19.958521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc9c451-1ba4-4b5e-b799-d7f99df29875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:19.985509 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-6cc9c451-1ba4-4b5e-b799-d7f99df29875 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:19.986764 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6cc9c451-1ba4-4b5e-b799-d7f99df29875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:20.060214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1314/2606] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:20.070755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a17ada-c4bd-4599-8eb1-0b4b83f21b1a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:20.071453 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a17ada-c4bd-4599-8eb1-0b4b83f21b1a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:20.074099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a17ada-c4bd-4599-8eb1-0b4b83f21b1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:20.102966 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-89a17ada-c4bd-4599-8eb1-0b4b83f21b1a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:20.105065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-89a17ada-c4bd-4599-8eb1-0b4b83f21b1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:20.114727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1293/2607] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:20.119125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9c451-1ba4-4b5e-b799-d7f99df29875 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:20.130917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc9c451-1ba4-4b5e-b799-d7f99df29875 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:20.135740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf438e-70ce-4ae0-af74-116417526615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:20.136362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf438e-70ce-4ae0-af74-116417526615 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:20.136641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf438e-70ce-4ae0-af74-116417526615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:20.136964 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eaf438e-70ce-4ae0-af74-116417526615 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:20.137618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf438e-70ce-4ae0-af74-116417526615 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:49:20.137921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf438e-70ce-4ae0-af74-116417526615 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:20.154683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eaf438e-70ce-4ae0-af74-116417526615 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:20.173591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1315/2608] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:49:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:49:20.177853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a17ada-c4bd-4599-8eb1-0b4b83f21b1a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:20.190095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a17ada-c4bd-4599-8eb1-0b4b83f21b1a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:20.195630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadc397-765a-49b9-b82d-184ac9b65e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:20.196241 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadc397-765a-49b9-b82d-184ac9b65e00 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:20.196542 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadc397-765a-49b9-b82d-184ac9b65e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:20.196840 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadc397-765a-49b9-b82d-184ac9b65e00 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:20.197482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadc397-765a-49b9-b82d-184ac9b65e00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:20.200038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadc397-765a-49b9-b82d-184ac9b65e00 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:20.255212 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1294/2609] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:20 2022] POST /identity/v3/projects => generated 363 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:20.259274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf438e-70ce-4ae0-af74-116417526615 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:20.272222 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eaf438e-70ce-4ae0-af74-116417526615 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:20.365019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f274ddd-12f5-4b17-b037-0610e7767346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:20.365654 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274ddd-12f5-4b17-b037-0610e7767346 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:20.365995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274ddd-12f5-4b17-b037-0610e7767346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:20.366230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274ddd-12f5-4b17-b037-0610e7767346 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:20.366936 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f274ddd-12f5-4b17-b037-0610e7767346 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:20.370613 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f274ddd-12f5-4b17-b037-0610e7767346 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:20.400799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1316/2610] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:20 2022] POST /identity/v3/users => generated 282 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:49:20.406036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadc397-765a-49b9-b82d-184ac9b65e00 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:20.416260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadc397-765a-49b9-b82d-184ac9b65e00 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:20.422309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d0f02-718b-40ba-be5b-a2ce36a708c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:20.422941 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d0f02-718b-40ba-be5b-a2ce36a708c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:20.423362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d0f02-718b-40ba-be5b-a2ce36a708c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:20.423640 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d0f02-718b-40ba-be5b-a2ce36a708c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:20.424511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d0f02-718b-40ba-be5b-a2ce36a708c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:20.426221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d0f02-718b-40ba-be5b-a2ce36a708c1 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:20.440624 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1295/2611] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:20.442006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f274ddd-12f5-4b17-b037-0610e7767346 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:20.453160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f274ddd-12f5-4b17-b037-0610e7767346 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:20.458176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae26c04-7955-456f-91d8-84220190e696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:20.458787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae26c04-7955-456f-91d8-84220190e696 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:20.459064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae26c04-7955-456f-91d8-84220190e696 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:20.459317 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae26c04-7955-456f-91d8-84220190e696 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:20.459951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae26c04-7955-456f-91d8-84220190e696 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:20.461590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae26c04-7955-456f-91d8-84220190e696 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:20.473770 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1317/2612] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:20.478003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d0f02-718b-40ba-be5b-a2ce36a708c1 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:20.488232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d0f02-718b-40ba-be5b-a2ce36a708c1 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:20.494497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dbb4d-14ee-406a-a827-b3417f58d47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:20.496997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dbb4d-14ee-406a-a827-b3417f58d47d admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:20.496997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dbb4d-14ee-406a-a827-b3417f58d47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:20.496997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071dbb4d-14ee-406a-a827-b3417f58d47d admin admin] PATH_INFO: `/v3/projects/34631afbc91241c8922cbb57b7467b15/users/0902a86221ad4ef9b8f22217bc78e528/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:20.514787 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dbb4d-14ee-406a-a827-b3417f58d47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=34631afbc91241c8922cbb57b7467b15, user_id=0902a86221ad4ef9b8f22217bc78e528, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:20.517531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-071dbb4d-14ee-406a-a827-b3417f58d47d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:20.536574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1296/2613] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:20 2022] PUT /identity/v3/projects/34631afbc91241c8922cbb57b7467b15/users/0902a86221ad4ef9b8f22217bc78e528/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:20.547293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18759911-8acc-474f-a5bc-de4e8abd5ed5 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:20.547593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18759911-8acc-474f-a5bc-de4e8abd5ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:20.547880 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18759911-8acc-474f-a5bc-de4e8abd5ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:20.604304 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-18759911-8acc-474f-a5bc-de4e8abd5ed5 None None] MFA Rules not processed for user `0902a86221ad4ef9b8f22217bc78e528`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:20.605837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-18759911-8acc-474f-a5bc-de4e8abd5ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:20.610279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:20.690279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1318/2614] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:20.695615 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dbb4d-14ee-406a-a827-b3417f58d47d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:20.734430 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:20.784253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071dbb4d-14ee-406a-a827-b3417f58d47d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:20.789407 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c2655-ea4b-415c-9193-dbfddd88a201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:20.790011 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c2655-ea4b-415c-9193-dbfddd88a201 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:20.790288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c2655-ea4b-415c-9193-dbfddd88a201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:20.790556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728c2655-ea4b-415c-9193-dbfddd88a201 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:20.791183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c2655-ea4b-415c-9193-dbfddd88a201 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:20.793876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728c2655-ea4b-415c-9193-dbfddd88a201 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:20.821353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1297/2615] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:20 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:49:20.826010 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18759911-8acc-474f-a5bc-de4e8abd5ed5 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:20.847752 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18759911-8acc-474f-a5bc-de4e8abd5ed5 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:20.969676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e40863-d881-44cc-8389-f23eac251bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:20.971169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e40863-d881-44cc-8389-f23eac251bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:20.971443 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e40863-d881-44cc-8389-f23eac251bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:20.971710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e40863-d881-44cc-8389-f23eac251bb4 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:20.972338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e40863-d881-44cc-8389-f23eac251bb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:20.974296 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e40863-d881-44cc-8389-f23eac251bb4 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:21.007376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1319/2616] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:20 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:21.008751 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c2655-ea4b-415c-9193-dbfddd88a201 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:21.020082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728c2655-ea4b-415c-9193-dbfddd88a201 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:21.024982 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4531869-f578-4de2-84c7-cef4cf58177d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:21.025590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4531869-f578-4de2-84c7-cef4cf58177d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:21.025890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4531869-f578-4de2-84c7-cef4cf58177d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:21.027343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4531869-f578-4de2-84c7-cef4cf58177d admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:21.028007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4531869-f578-4de2-84c7-cef4cf58177d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:21.030067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4531869-f578-4de2-84c7-cef4cf58177d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:21.046379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1298/2617] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:49:21.047857 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e40863-d881-44cc-8389-f23eac251bb4 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:21.059788 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e40863-d881-44cc-8389-f23eac251bb4 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:21.065004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c02ef6-24e0-4308-b099-1fb0520add95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:21.065834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02ef6-24e0-4308-b099-1fb0520add95 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:21.066118 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02ef6-24e0-4308-b099-1fb0520add95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:21.066391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c02ef6-24e0-4308-b099-1fb0520add95 admin admin] PATH_INFO: `/v3/projects/822bc0bc974b46fab3009f730db3f07f/users/6a15d64845794d46b5e558dfb5dd6cd8/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:21.085754 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c02ef6-24e0-4308-b099-1fb0520add95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=822bc0bc974b46fab3009f730db3f07f, user_id=6a15d64845794d46b5e558dfb5dd6cd8, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:21.095707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c02ef6-24e0-4308-b099-1fb0520add95 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:21.115913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1320/2618] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:21 2022] PUT /identity/v3/projects/822bc0bc974b46fab3009f730db3f07f/users/6a15d64845794d46b5e558dfb5dd6cd8/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:49:21.120333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4531869-f578-4de2-84c7-cef4cf58177d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:21.130971 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4531869-f578-4de2-84c7-cef4cf58177d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:21.197105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f9b9bb-ac18-4dff-94ed-630a77f21077 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:21.197639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9b9bb-ac18-4dff-94ed-630a77f21077 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:21.198364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9b9bb-ac18-4dff-94ed-630a77f21077 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:21.198364 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f9b9bb-ac18-4dff-94ed-630a77f21077 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:21.198850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9b9bb-ac18-4dff-94ed-630a77f21077 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:21.200516 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f9b9bb-ac18-4dff-94ed-630a77f21077 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:21.213630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1299/2619] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:21 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:21.219055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c02ef6-24e0-4308-b099-1fb0520add95 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:21.231855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c02ef6-24e0-4308-b099-1fb0520add95 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:21.238339 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2b23f-867f-425c-9c7e-20cb0556e365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:21.239980 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2b23f-867f-425c-9c7e-20cb0556e365 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:21.240334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2b23f-867f-425c-9c7e-20cb0556e365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:21.240742 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe2b23f-867f-425c-9c7e-20cb0556e365 admin admin] PATH_INFO: `/v3/projects/822bc0bc974b46fab3009f730db3f07f/users/6a15d64845794d46b5e558dfb5dd6cd8/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:21.243110 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2b23f-867f-425c-9c7e-20cb0556e365 admin admin] RBAC: Authorizing `identity:create_grant(project_id=822bc0bc974b46fab3009f730db3f07f, user_id=6a15d64845794d46b5e558dfb5dd6cd8, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:21.246928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe2b23f-867f-425c-9c7e-20cb0556e365 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:21.263295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1321/2620] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:21 2022] PUT /identity/v3/projects/822bc0bc974b46fab3009f730db3f07f/users/6a15d64845794d46b5e558dfb5dd6cd8/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:49:21.278936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6d452-3b55-414e-a7a7-d5a922e0ee60 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:21.279251 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6d452-3b55-414e-a7a7-d5a922e0ee60 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:21.279498 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f6d452-3b55-414e-a7a7-d5a922e0ee60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:21.323836 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-95f6d452-3b55-414e-a7a7-d5a922e0ee60 None None] MFA Rules not processed for user `6a15d64845794d46b5e558dfb5dd6cd8`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:21.325173 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-95f6d452-3b55-414e-a7a7-d5a922e0ee60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:21.400623 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1300/2621] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:21 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:21.414892 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2b23f-867f-425c-9c7e-20cb0556e365 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:21.416804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-ffe2b23f-867f-425c-9c7e-20cb0556e365 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:21.485396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe2b23f-867f-425c-9c7e-20cb0556e365 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:21.489500 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8932f2-f915-4af7-a6d1-7536836a12a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:21.490362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8932f2-f915-4af7-a6d1-7536836a12a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:21.490850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8932f2-f915-4af7-a6d1-7536836a12a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:21.491125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8932f2-f915-4af7-a6d1-7536836a12a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:21.502502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8932f2-f915-4af7-a6d1-7536836a12a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:21.503930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8932f2-f915-4af7-a6d1-7536836a12a5 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:21.509088 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:21.515943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1322/2622] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:21 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:21.599204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6d452-3b55-414e-a7a7-d5a922e0ee60 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:21.610598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f6d452-3b55-414e-a7a7-d5a922e0ee60 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:21.615248 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62026a12-5afe-4606-b840-085936973eea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:21.615844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62026a12-5afe-4606-b840-085936973eea service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:21.616114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62026a12-5afe-4606-b840-085936973eea service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:21.616378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62026a12-5afe-4606-b840-085936973eea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:21.625588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:21.674895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62026a12-5afe-4606-b840-085936973eea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:21.676340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62026a12-5afe-4606-b840-085936973eea service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:21.685588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1301/2623] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:21 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:24.481466 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8932f2-f915-4af7-a6d1-7536836a12a5 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:24.568657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8932f2-f915-4af7-a6d1-7536836a12a5 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:24.585537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:24.586512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:24.586512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:24.587265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] PATH_INFO: `/v3/users/0902a86221ad4ef9b8f22217bc78e528` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:24.588548 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0902a86221ad4ef9b8f22217bc78e528)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:24.591197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:24.649498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0902a86221ad4ef9b8f22217bc78e528'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:24.659328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0902a86221ad4ef9b8f22217bc78e528'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:24.660105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0902a86221ad4ef9b8f22217bc78e528'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:24.678962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0902a86221ad4ef9b8f22217bc78e528'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:24.687317 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0902a86221ad4ef9b8f22217bc78e528'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:24.697163 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1323/2624] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:24 2022] DELETE /identity/v3/users/0902a86221ad4ef9b8f22217bc78e528 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:24.699543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c34cd-4f86-4275-bb00-37a6bf7aadee None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:24.700064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c34cd-4f86-4275-bb00-37a6bf7aadee None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:24.700064 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987c34cd-4f86-4275-bb00-37a6bf7aadee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:24.736231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-987c34cd-4f86-4275-bb00-37a6bf7aadee None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:24.737524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-987c34cd-4f86-4275-bb00-37a6bf7aadee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:24.803284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1302/2625] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:24.817488 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:24.818908 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:24.910627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c387ce6a-c647-46bc-b14d-d03896232d41 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:24.921560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618566a1-78d7-4b35-b275-2dc65c8be127 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:24.922201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618566a1-78d7-4b35-b275-2dc65c8be127 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:24.922476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618566a1-78d7-4b35-b275-2dc65c8be127 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:24.922747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618566a1-78d7-4b35-b275-2dc65c8be127 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:24.938159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618566a1-78d7-4b35-b275-2dc65c8be127 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:24.939575 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618566a1-78d7-4b35-b275-2dc65c8be127 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:24.955030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1324/2626] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:25.265123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c34cd-4f86-4275-bb00-37a6bf7aadee None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:25.266655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-987c34cd-4f86-4275-bb00-37a6bf7aadee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:25.286629 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987c34cd-4f86-4275-bb00-37a6bf7aadee None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:25.292413 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:25.293087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:25.293360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:25.293630 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] PATH_INFO: `/v3/projects/34631afbc91241c8922cbb57b7467b15` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:25.294785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] RBAC: Authorizing `identity:delete_project(project_id=34631afbc91241c8922cbb57b7467b15)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:25.296695 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:25.367850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34631afbc91241c8922cbb57b7467b15', 'request_id': 'req-3a5bad9f-655c-40f0-b857-798a96761eec'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:25.379101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34631afbc91241c8922cbb57b7467b15', 'request_id': 'req-3a5bad9f-655c-40f0-b857-798a96761eec'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:25.389355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34631afbc91241c8922cbb57b7467b15', 'request_id': 'req-3a5bad9f-655c-40f0-b857-798a96761eec'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:25.400296 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] The token cache is being invalidate because project 34631afbc91241c8922cbb57b7467b15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:25.400581 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:25.401387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:25.403144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1303/2627] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:25 2022] DELETE /identity/v3/projects/34631afbc91241c8922cbb57b7467b15 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:25.407203 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618566a1-78d7-4b35-b275-2dc65c8be127 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:25.418477 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618566a1-78d7-4b35-b275-2dc65c8be127 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:25.440245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:25.487737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4331db5a-11e5-4df3-9175-4720f6ffc2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:25.488452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331db5a-11e5-4df3-9175-4720f6ffc2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:25.488731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331db5a-11e5-4df3-9175-4720f6ffc2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:25.489029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4331db5a-11e5-4df3-9175-4720f6ffc2f0 admin admin] PATH_INFO: `/v3/users/6a15d64845794d46b5e558dfb5dd6cd8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:25.490180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331db5a-11e5-4df3-9175-4720f6ffc2f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a15d64845794d46b5e558dfb5dd6cd8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:25.492316 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4331db5a-11e5-4df3-9175-4720f6ffc2f0 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:25.543125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4331db5a-11e5-4df3-9175-4720f6ffc2f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a15d64845794d46b5e558dfb5dd6cd8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:25.556271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4331db5a-11e5-4df3-9175-4720f6ffc2f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a15d64845794d46b5e558dfb5dd6cd8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:25.557009 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4331db5a-11e5-4df3-9175-4720f6ffc2f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a15d64845794d46b5e558dfb5dd6cd8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:25.571623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4331db5a-11e5-4df3-9175-4720f6ffc2f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a15d64845794d46b5e558dfb5dd6cd8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:25.580573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-4331db5a-11e5-4df3-9175-4720f6ffc2f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a15d64845794d46b5e558dfb5dd6cd8'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:25.587378 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1325/2628] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:25 2022] DELETE /identity/v3/users/6a15d64845794d46b5e558dfb5dd6cd8 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:25.889963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:25.894264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:25.921708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5bad9f-655c-40f0-b857-798a96761eec admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:25.977644 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:26.020136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:26.021224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:26.021698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:26.022181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] PATH_INFO: `/v3/projects/822bc0bc974b46fab3009f730db3f07f` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:26.023549 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] RBAC: Authorizing `identity:delete_project(project_id=822bc0bc974b46fab3009f730db3f07f)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:26.026833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:26.105374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '822bc0bc974b46fab3009f730db3f07f', 'request_id': 'req-b61bf54a-1af0-48c9-88a4-4e6322c88718'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:26.130319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '822bc0bc974b46fab3009f730db3f07f', 'request_id': 'req-b61bf54a-1af0-48c9-88a4-4e6322c88718'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:26.154409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '822bc0bc974b46fab3009f730db3f07f', 'request_id': 'req-b61bf54a-1af0-48c9-88a4-4e6322c88718'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:26.171924 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] The token cache is being invalidate because project 822bc0bc974b46fab3009f730db3f07f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:26.172120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:26.173341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-b61bf54a-1af0-48c9-88a4-4e6322c88718 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:26.175501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1304/2629] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:25 2022] DELETE /identity/v3/projects/822bc0bc974b46fab3009f730db3f07f => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:26.180397 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f674c-1c4c-40ca-8c46-24dfd35e5709 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:26.181002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f674c-1c4c-40ca-8c46-24dfd35e5709 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:26.181531 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f674c-1c4c-40ca-8c46-24dfd35e5709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:26.234122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-958f674c-1c4c-40ca-8c46-24dfd35e5709 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:26.239386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-958f674c-1c4c-40ca-8c46-24dfd35e5709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:26.419336 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1326/2630] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:26.426994 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539c205-e6ad-4355-8dc4-7c6fbe8342b6 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:26.427284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539c205-e6ad-4355-8dc4-7c6fbe8342b6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:26.427565 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a539c205-e6ad-4355-8dc4-7c6fbe8342b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:26.465177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a539c205-e6ad-4355-8dc4-7c6fbe8342b6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:26.466035 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a539c205-e6ad-4355-8dc4-7c6fbe8342b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:26.475876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1305/2631] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:26.481629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f674c-1c4c-40ca-8c46-24dfd35e5709 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:26.493549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958f674c-1c4c-40ca-8c46-24dfd35e5709 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:26.499233 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b4668-0553-404b-9feb-abfa28d2a0c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:26.499861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b4668-0553-404b-9feb-abfa28d2a0c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:26.500135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b4668-0553-404b-9feb-abfa28d2a0c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:26.500404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b4668-0553-404b-9feb-abfa28d2a0c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:26.501091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b4668-0553-404b-9feb-abfa28d2a0c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:49:26.501391 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b4668-0553-404b-9feb-abfa28d2a0c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:26.503609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b4668-0553-404b-9feb-abfa28d2a0c7 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:26.522559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1327/2632] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:49:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:49:26.523092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539c205-e6ad-4355-8dc4-7c6fbe8342b6 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:26.536402 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a539c205-e6ad-4355-8dc4-7c6fbe8342b6 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:26.542358 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62799e70-e4e9-4a4a-94ea-27bdc0fcc598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:26.542986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62799e70-e4e9-4a4a-94ea-27bdc0fcc598 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:26.543263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62799e70-e4e9-4a4a-94ea-27bdc0fcc598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:26.543533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62799e70-e4e9-4a4a-94ea-27bdc0fcc598 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:26.544490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62799e70-e4e9-4a4a-94ea-27bdc0fcc598 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:26.546960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62799e70-e4e9-4a4a-94ea-27bdc0fcc598 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:26.597845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1306/2633] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:26 2022] POST /identity/v3/projects => generated 379 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:26.600465 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b4668-0553-404b-9feb-abfa28d2a0c7 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:26.614899 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b4668-0553-404b-9feb-abfa28d2a0c7 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:26.650807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:26.692291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b8bb2-69d3-4f1e-8507-063e9b7d1b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:26.693065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b8bb2-69d3-4f1e-8507-063e9b7d1b48 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:26.693351 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b8bb2-69d3-4f1e-8507-063e9b7d1b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:26.693629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180b8bb2-69d3-4f1e-8507-063e9b7d1b48 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:26.694308 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b8bb2-69d3-4f1e-8507-063e9b7d1b48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:26.696204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180b8bb2-69d3-4f1e-8507-063e9b7d1b48 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:26.722159 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1328/2634] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:26 2022] POST /identity/v3/users => generated 290 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:26.726149 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62799e70-e4e9-4a4a-94ea-27bdc0fcc598 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:26.738217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62799e70-e4e9-4a4a-94ea-27bdc0fcc598 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:26.746845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c87028-e7e4-4f38-8220-0f7157c8d1b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:26.747460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c87028-e7e4-4f38-8220-0f7157c8d1b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:26.747734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c87028-e7e4-4f38-8220-0f7157c8d1b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:26.748006 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c87028-e7e4-4f38-8220-0f7157c8d1b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:26.748653 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c87028-e7e4-4f38-8220-0f7157c8d1b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:26.750272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c87028-e7e4-4f38-8220-0f7157c8d1b8 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:26.766534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b8bb2-69d3-4f1e-8507-063e9b7d1b48 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:26.767544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1307/2635] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:26.778805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180b8bb2-69d3-4f1e-8507-063e9b7d1b48 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:26.785118 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f21c4c0-afc0-4d10-b2e8-9ad8c861fb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:26.785915 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21c4c0-afc0-4d10-b2e8-9ad8c861fb70 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:26.786197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21c4c0-afc0-4d10-b2e8-9ad8c861fb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:26.786473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f21c4c0-afc0-4d10-b2e8-9ad8c861fb70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:26.787324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21c4c0-afc0-4d10-b2e8-9ad8c861fb70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:26.788732 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f21c4c0-afc0-4d10-b2e8-9ad8c861fb70 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:26.804091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1329/2636] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:26 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:26.805419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c87028-e7e4-4f38-8220-0f7157c8d1b8 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:26.816040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c87028-e7e4-4f38-8220-0f7157c8d1b8 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:26.821633 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1713e-433d-42cf-b70a-6f21e92fa18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:26.822488 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1713e-433d-42cf-b70a-6f21e92fa18b admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:26.822969 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1713e-433d-42cf-b70a-6f21e92fa18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:26.823394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1713e-433d-42cf-b70a-6f21e92fa18b admin admin] PATH_INFO: `/v3/projects/012b2a48e5e045d1985bb7ad3b09a2f5/users/01bec8e6d3774e858909fa9b6e8f9e62/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:26.844257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1713e-433d-42cf-b70a-6f21e92fa18b admin admin] RBAC: Authorizing `identity:create_grant(project_id=012b2a48e5e045d1985bb7ad3b09a2f5, user_id=01bec8e6d3774e858909fa9b6e8f9e62, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:26.847653 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1713e-433d-42cf-b70a-6f21e92fa18b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:26.878552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1308/2637] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:26 2022] PUT /identity/v3/projects/012b2a48e5e045d1985bb7ad3b09a2f5/users/01bec8e6d3774e858909fa9b6e8f9e62/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:49:26.881006 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc36a4-0164-4817-8f0f-8272454cb028 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:26.882810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc36a4-0164-4817-8f0f-8272454cb028 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:26.883270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dc36a4-0164-4817-8f0f-8272454cb028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:26.936818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-37dc36a4-0164-4817-8f0f-8272454cb028 None None] MFA Rules not processed for user `01bec8e6d3774e858909fa9b6e8f9e62`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:26.938346 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-37dc36a4-0164-4817-8f0f-8272454cb028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:27.027701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1330/2638] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:27.032302 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1713e-433d-42cf-b70a-6f21e92fa18b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:27.108009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1713e-433d-42cf-b70a-6f21e92fa18b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:27.113204 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2ee29-846c-41cd-8a27-394678a81bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:27.113960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2ee29-846c-41cd-8a27-394678a81bbe admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:27.114233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2ee29-846c-41cd-8a27-394678a81bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:27.114575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa2ee29-846c-41cd-8a27-394678a81bbe admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:27.115715 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2ee29-846c-41cd-8a27-394678a81bbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:27.117578 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa2ee29-846c-41cd-8a27-394678a81bbe admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:27.129008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:27.191156 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1309/2639] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:27 2022] POST /identity/v3/projects => generated 381 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:27.195658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc36a4-0164-4817-8f0f-8272454cb028 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:27.207917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dc36a4-0164-4817-8f0f-8272454cb028 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:27.274448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956daa2-8557-49f7-9c83-63b25ddcca4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:27.275015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956daa2-8557-49f7-9c83-63b25ddcca4e admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:27.275294 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956daa2-8557-49f7-9c83-63b25ddcca4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:27.275562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956daa2-8557-49f7-9c83-63b25ddcca4e admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:27.278015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956daa2-8557-49f7-9c83-63b25ddcca4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:27.280123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956daa2-8557-49f7-9c83-63b25ddcca4e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:27.305570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1331/2640] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:27 2022] POST /identity/v3/users => generated 297 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:49:27.310333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2ee29-846c-41cd-8a27-394678a81bbe admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:27.320990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa2ee29-846c-41cd-8a27-394678a81bbe admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:27.327076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4896dca-514f-4f53-aafc-af007cb0af02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:27.328327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4896dca-514f-4f53-aafc-af007cb0af02 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:27.328327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4896dca-514f-4f53-aafc-af007cb0af02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:27.328327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4896dca-514f-4f53-aafc-af007cb0af02 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:27.329044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4896dca-514f-4f53-aafc-af007cb0af02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:27.330685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4896dca-514f-4f53-aafc-af007cb0af02 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:27.341658 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1310/2641] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:27.345979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956daa2-8557-49f7-9c83-63b25ddcca4e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:27.356974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956daa2-8557-49f7-9c83-63b25ddcca4e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:27.362223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3662d75-5d8e-4492-9877-ba08fc78a716 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:27.362705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3662d75-5d8e-4492-9877-ba08fc78a716 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:27.362981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3662d75-5d8e-4492-9877-ba08fc78a716 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:27.363263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3662d75-5d8e-4492-9877-ba08fc78a716 admin admin] PATH_INFO: `/v3/projects/d792fc0301964a3386d04a011290e7e7/users/14fe6fe2d21c48458b5c24504d3a08ed/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:27.380623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3662d75-5d8e-4492-9877-ba08fc78a716 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d792fc0301964a3386d04a011290e7e7, user_id=14fe6fe2d21c48458b5c24504d3a08ed, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:27.382980 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3662d75-5d8e-4492-9877-ba08fc78a716 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:27.396976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1332/2642] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:27 2022] PUT /identity/v3/projects/d792fc0301964a3386d04a011290e7e7/users/14fe6fe2d21c48458b5c24504d3a08ed/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:27.401935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4896dca-514f-4f53-aafc-af007cb0af02 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:27.412166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4896dca-514f-4f53-aafc-af007cb0af02 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:27.470118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4908c5-9c85-4146-8646-f5caeb7b6a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:27.470724 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4908c5-9c85-4146-8646-f5caeb7b6a8c admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:27.471003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4908c5-9c85-4146-8646-f5caeb7b6a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:27.471272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4908c5-9c85-4146-8646-f5caeb7b6a8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:27.471924 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4908c5-9c85-4146-8646-f5caeb7b6a8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:27.473528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4908c5-9c85-4146-8646-f5caeb7b6a8c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:27.484391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1311/2643] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:27 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:27.488473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3662d75-5d8e-4492-9877-ba08fc78a716 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:27.493980 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:27.500669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3662d75-5d8e-4492-9877-ba08fc78a716 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:27.507231 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a49a3c-e793-46d6-9a0b-c3dbb97a8d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:27.507893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a49a3c-e793-46d6-9a0b-c3dbb97a8d86 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:27.508173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a49a3c-e793-46d6-9a0b-c3dbb97a8d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:27.508444 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a49a3c-e793-46d6-9a0b-c3dbb97a8d86 admin admin] PATH_INFO: `/v3/projects/d792fc0301964a3386d04a011290e7e7/users/14fe6fe2d21c48458b5c24504d3a08ed/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:27.510459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a49a3c-e793-46d6-9a0b-c3dbb97a8d86 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d792fc0301964a3386d04a011290e7e7, user_id=14fe6fe2d21c48458b5c24504d3a08ed, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:27.513283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a49a3c-e793-46d6-9a0b-c3dbb97a8d86 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:27.523786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1333/2644] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:27 2022] PUT /identity/v3/projects/d792fc0301964a3386d04a011290e7e7/users/14fe6fe2d21c48458b5c24504d3a08ed/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:49:27.536111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e75a7-df0d-4e6d-bb75-31acdce61dc1 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:27.536327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e75a7-df0d-4e6d-bb75-31acdce61dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:27.536600 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183e75a7-df0d-4e6d-bb75-31acdce61dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:27.566210 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-183e75a7-df0d-4e6d-bb75-31acdce61dc1 None None] MFA Rules not processed for user `14fe6fe2d21c48458b5c24504d3a08ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:27.567415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-183e75a7-df0d-4e6d-bb75-31acdce61dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:27.646194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1312/2645] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:27.659534 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a49a3c-e793-46d6-9a0b-c3dbb97a8d86 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:27.661278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-08a49a3c-e793-46d6-9a0b-c3dbb97a8d86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:27.853372 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a49a3c-e793-46d6-9a0b-c3dbb97a8d86 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:27.857490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15eabc-562c-4325-9deb-b986b994afd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:27.858102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15eabc-562c-4325-9deb-b986b994afd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:27.858374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15eabc-562c-4325-9deb-b986b994afd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:27.858640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15eabc-562c-4325-9deb-b986b994afd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:27.868227 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15eabc-562c-4325-9deb-b986b994afd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:27.870480 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15eabc-562c-4325-9deb-b986b994afd7 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:27.893335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1334/2646] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:27 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:28.411775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e75a7-df0d-4e6d-bb75-31acdce61dc1 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:28.426278 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183e75a7-df0d-4e6d-bb75-31acdce61dc1 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:28.434223 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e288f-7fa0-4f41-9266-4dee89fc33b6 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14fe6fe2d21c48458b5c24504d3a08ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd792fc0301964a3386d04a011290e7e7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:28.435138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e288f-7fa0-4f41-9266-4dee89fc33b6 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:28.435925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e288f-7fa0-4f41-9266-4dee89fc33b6 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:28.436357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e288f-7fa0-4f41-9266-4dee89fc33b6 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:28.437341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e288f-7fa0-4f41-9266-4dee89fc33b6 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:49:28.438118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e288f-7fa0-4f41-9266-4dee89fc33b6 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:28.440446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e288f-7fa0-4f41-9266-4dee89fc33b6 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:28.461994 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1313/2647] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:49:28.462245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15eabc-562c-4325-9deb-b986b994afd7 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:28.474669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15eabc-562c-4325-9deb-b986b994afd7 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:28.480342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4dedc-a4cf-482a-80d5-95ec4432efe0 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14fe6fe2d21c48458b5c24504d3a08ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd792fc0301964a3386d04a011290e7e7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:28.480753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4dedc-a4cf-482a-80d5-95ec4432efe0 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:28.481562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4dedc-a4cf-482a-80d5-95ec4432efe0 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:28.481562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4dedc-a4cf-482a-80d5-95ec4432efe0 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:28.482137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4dedc-a4cf-482a-80d5-95ec4432efe0 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:28.484087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4dedc-a4cf-482a-80d5-95ec4432efe0 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:28.524696 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1335/2648] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:28 2022] POST /identity/v3/projects => generated 355 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:28.527538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e288f-7fa0-4f41-9266-4dee89fc33b6 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:28.543144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e288f-7fa0-4f41-9266-4dee89fc33b6 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:28.632792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:28.662988 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a53f56-3152-4c49-b35c-9a47d6d1e1cf tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14fe6fe2d21c48458b5c24504d3a08ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd792fc0301964a3386d04a011290e7e7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:28.663958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a53f56-3152-4c49-b35c-9a47d6d1e1cf tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:28.664245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a53f56-3152-4c49-b35c-9a47d6d1e1cf tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:28.664568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a53f56-3152-4c49-b35c-9a47d6d1e1cf tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:28.665206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a53f56-3152-4c49-b35c-9a47d6d1e1cf tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:49:28.665511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a53f56-3152-4c49-b35c-9a47d6d1e1cf tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:28.667300 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a53f56-3152-4c49-b35c-9a47d6d1e1cf tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:28.681627 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1314/2649] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:49:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:49:28.721955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4dedc-a4cf-482a-80d5-95ec4432efe0 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:28.732407 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4dedc-a4cf-482a-80d5-95ec4432efe0 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:28.737228 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '14fe6fe2d21c48458b5c24504d3a08ed', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd792fc0301964a3386d04a011290e7e7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:28.737964 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:28.738305 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:28.738638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] PATH_INFO: `/v3/projects/40fae22320bb450fa986942faf5694ed` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:28.739835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: Authorizing `identity:delete_project(project_id=40fae22320bb450fa986942faf5694ed)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:28.741881 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:28.792631 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40fae22320bb450fa986942faf5694ed', 'request_id': 'req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:28.800718 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40fae22320bb450fa986942faf5694ed', 'request_id': 'req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:28.806357 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40fae22320bb450fa986942faf5694ed', 'request_id': 'req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:28.818521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] The token cache is being invalidate because project 40fae22320bb450fa986942faf5694ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:28.818800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:28.819458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e9e523b4-8b94-4fc7-be7d-3f6d1c3d3c18 tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:28.820972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1336/2650] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:28 2022] DELETE /identity/v3/projects/40fae22320bb450fa986942faf5694ed => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:29.038791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a53f56-3152-4c49-b35c-9a47d6d1e1cf tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:29.040568 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d1a53f56-3152-4c49-b35c-9a47d6d1e1cf tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:29.109284 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a53f56-3152-4c49-b35c-9a47d6d1e1cf tempest-VolumeQuotaClassesTest-1129330068 tempest-VolumeQuotaClassesTest-1129330068-project-admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:29.167813 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:29.168550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:29.168933 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:29.169202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] PATH_INFO: `/v3/users/01bec8e6d3774e858909fa9b6e8f9e62` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:29.170392 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01bec8e6d3774e858909fa9b6e8f9e62)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:29.172352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:29.222342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01bec8e6d3774e858909fa9b6e8f9e62'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:29.222986 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01bec8e6d3774e858909fa9b6e8f9e62'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:29.231404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01bec8e6d3774e858909fa9b6e8f9e62'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:29.238918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01bec8e6d3774e858909fa9b6e8f9e62'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:29.252103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01bec8e6d3774e858909fa9b6e8f9e62'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:29.262791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1315/2651] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:29 2022] DELETE /identity/v3/users/01bec8e6d3774e858909fa9b6e8f9e62 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:29.264979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be160eda-60c0-4cb8-977d-2121c94b77bd None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:29.265263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be160eda-60c0-4cb8-977d-2121c94b77bd None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:29.265590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be160eda-60c0-4cb8-977d-2121c94b77bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:29.286661 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-be160eda-60c0-4cb8-977d-2121c94b77bd None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:29.287912 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-be160eda-60c0-4cb8-977d-2121c94b77bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:29.347061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:29.366875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1337/2652] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:29.378157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:29.379777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:29.385774 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:29.449915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25bbba-c27b-4ade-98dd-eb2ba7109610 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:29.453611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5032a9-064b-4d30-9b59-8ac9dda4615d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:29.454215 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5032a9-064b-4d30-9b59-8ac9dda4615d service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:29.454490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5032a9-064b-4d30-9b59-8ac9dda4615d service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:29.454764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5032a9-064b-4d30-9b59-8ac9dda4615d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:29.464602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5032a9-064b-4d30-9b59-8ac9dda4615d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:29.466059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5032a9-064b-4d30-9b59-8ac9dda4615d service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:29.475594 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1316/2653] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:29.830039 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be160eda-60c0-4cb8-977d-2121c94b77bd None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:29.831469 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-be160eda-60c0-4cb8-977d-2121c94b77bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:29.843283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be160eda-60c0-4cb8-977d-2121c94b77bd None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:29.847800 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:29.848427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:29.848697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:29.849001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] PATH_INFO: `/v3/projects/012b2a48e5e045d1985bb7ad3b09a2f5` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:29.850107 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=012b2a48e5e045d1985bb7ad3b09a2f5)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:29.851956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:29.900766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '012b2a48e5e045d1985bb7ad3b09a2f5', 'request_id': 'req-92987b7d-8aa8-4986-b7c7-5bda2c079e56'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:29.908753 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '012b2a48e5e045d1985bb7ad3b09a2f5', 'request_id': 'req-92987b7d-8aa8-4986-b7c7-5bda2c079e56'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:29.915129 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '012b2a48e5e045d1985bb7ad3b09a2f5', 'request_id': 'req-92987b7d-8aa8-4986-b7c7-5bda2c079e56'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:29.927091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] The token cache is being invalidate because project 012b2a48e5e045d1985bb7ad3b09a2f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:29.927373 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:29.928023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-92987b7d-8aa8-4986-b7c7-5bda2c079e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:29.929556 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1338/2654] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:29 2022] DELETE /identity/v3/projects/012b2a48e5e045d1985bb7ad3b09a2f5 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:29.934404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5032a9-064b-4d30-9b59-8ac9dda4615d service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:29.944595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5032a9-064b-4d30-9b59-8ac9dda4615d service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:30.014746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:30.015672 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:30.016069 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:30.016439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] PATH_INFO: `/v3/users/14fe6fe2d21c48458b5c24504d3a08ed` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:30.018163 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] RBAC: Authorizing `identity:delete_user(user_id=14fe6fe2d21c48458b5c24504d3a08ed)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:30.020261 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:30.079719 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '14fe6fe2d21c48458b5c24504d3a08ed'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:30.080412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '14fe6fe2d21c48458b5c24504d3a08ed'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:30.091086 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '14fe6fe2d21c48458b5c24504d3a08ed'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:30.099866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '14fe6fe2d21c48458b5c24504d3a08ed'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:30.114659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '14fe6fe2d21c48458b5c24504d3a08ed'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:30.125144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1317/2655] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:29 2022] DELETE /identity/v3/users/14fe6fe2d21c48458b5c24504d3a08ed => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:30.378931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503692a8-8c50-4c3a-a600-ef62b9699d0f None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:30.379297 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503692a8-8c50-4c3a-a600-ef62b9699d0f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:30.379893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503692a8-8c50-4c3a-a600-ef62b9699d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:30.407799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-503692a8-8c50-4c3a-a600-ef62b9699d0f None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:30.409639 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-503692a8-8c50-4c3a-a600-ef62b9699d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:30.431203 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:30.433493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:30.484099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:30.499708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1339/2656] 10.222.0.39 () {58 vars in 967 bytes} [Tue Sep 6 22:49:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:30.507549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503692a8-8c50-4c3a-a600-ef62b9699d0f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:30.509151 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-503692a8-8c50-4c3a-a600-ef62b9699d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:30.546838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119dbb5-ad50-404b-b68e-0be11070e86a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:30.553509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:30.554159 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:30.554441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:30.554710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] PATH_INFO: `/v3/projects/d792fc0301964a3386d04a011290e7e7` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:30.555861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d792fc0301964a3386d04a011290e7e7)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:30.560319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:30.597062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503692a8-8c50-4c3a-a600-ef62b9699d0f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:30.608324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b82aaa-06a9-4168-b48e-e2aee3b16c50 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:30.608995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b82aaa-06a9-4168-b48e-e2aee3b16c50 service swift] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:30.609271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b82aaa-06a9-4168-b48e-e2aee3b16c50 service swift] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:30.609538 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b82aaa-06a9-4168-b48e-e2aee3b16c50 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:30.622147 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd792fc0301964a3386d04a011290e7e7', 'request_id': 'req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:30.626026 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b82aaa-06a9-4168-b48e-e2aee3b16c50 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:30.627596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b82aaa-06a9-4168-b48e-e2aee3b16c50 service swift] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:30.629505 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd792fc0301964a3386d04a011290e7e7', 'request_id': 'req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:30.639801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1340/2657] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:49:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 22:49:30.640206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd792fc0301964a3386d04a011290e7e7', 'request_id': 'req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:30.655785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] The token cache is being invalidate because project d792fc0301964a3386d04a011290e7e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:30.656078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:30.656730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-2b4a1c64-96ad-44f0-a6ae-605666d9ab67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:30.658379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1318/2658] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:30 2022] DELETE /identity/v3/projects/d792fc0301964a3386d04a011290e7e7 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:49:30.663298 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ca0b1-9233-4471-acd3-57365c11b383 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:30.663647 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ca0b1-9233-4471-acd3-57365c11b383 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:30.663919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95ca0b1-9233-4471-acd3-57365c11b383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:30.685034 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c95ca0b1-9233-4471-acd3-57365c11b383 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:30.686216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c95ca0b1-9233-4471-acd3-57365c11b383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:30.691553 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:30.750259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1341/2659] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:30.758060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e20b38-1fc2-40ee-9d32-04f6d7cf1b7b None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:30.758343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e20b38-1fc2-40ee-9d32-04f6d7cf1b7b None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:30.758617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e20b38-1fc2-40ee-9d32-04f6d7cf1b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:30.779913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c3e20b38-1fc2-40ee-9d32-04f6d7cf1b7b None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:30.781229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c3e20b38-1fc2-40ee-9d32-04f6d7cf1b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:30.789794 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1319/2660] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:30.794493 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ca0b1-9233-4471-acd3-57365c11b383 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:30.804665 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95ca0b1-9233-4471-acd3-57365c11b383 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:30.810239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1bc3c-3dd7-434b-aaaa-2c0160731341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:30.810848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1bc3c-3dd7-434b-aaaa-2c0160731341 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:30.811120 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1bc3c-3dd7-434b-aaaa-2c0160731341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:30.811509 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1bc3c-3dd7-434b-aaaa-2c0160731341 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:30.812160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1bc3c-3dd7-434b-aaaa-2c0160731341 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:49:30.812461 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1bc3c-3dd7-434b-aaaa-2c0160731341 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:30.814080 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf1bc3c-3dd7-434b-aaaa-2c0160731341 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:30.832811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1342/2661] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:49:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:49:30.834902 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e20b38-1fc2-40ee-9d32-04f6d7cf1b7b None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:30.846557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e20b38-1fc2-40ee-9d32-04f6d7cf1b7b None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:30.851053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70938d3-5e01-48fa-910a-7e80dca625ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:30.851750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70938d3-5e01-48fa-910a-7e80dca625ae admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:30.852060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70938d3-5e01-48fa-910a-7e80dca625ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:30.852329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70938d3-5e01-48fa-910a-7e80dca625ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:30.853030 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70938d3-5e01-48fa-910a-7e80dca625ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:30.855493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70938d3-5e01-48fa-910a-7e80dca625ae admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:30.889357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1320/2662] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:30 2022] POST /identity/v3/projects => generated 409 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:30.893568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1bc3c-3dd7-434b-aaaa-2c0160731341 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:30.904683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1bc3c-3dd7-434b-aaaa-2c0160731341 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:30.972168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724cc4d-6438-418f-a76e-4b19f48a1785 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:30.972801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724cc4d-6438-418f-a76e-4b19f48a1785 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:30.973084 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724cc4d-6438-418f-a76e-4b19f48a1785 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:30.973355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1724cc4d-6438-418f-a76e-4b19f48a1785 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:30.973995 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724cc4d-6438-418f-a76e-4b19f48a1785 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:30.975890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1724cc4d-6438-418f-a76e-4b19f48a1785 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:31.004209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1343/2663] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:30 2022] POST /identity/v3/users => generated 305 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:49:31.005514 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70938d3-5e01-48fa-910a-7e80dca625ae admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:31.016290 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70938d3-5e01-48fa-910a-7e80dca625ae admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:31.021654 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce9642-bb88-45ec-b201-4af3bf47d365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:31.022247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce9642-bb88-45ec-b201-4af3bf47d365 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.022528 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce9642-bb88-45ec-b201-4af3bf47d365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.022855 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ce9642-bb88-45ec-b201-4af3bf47d365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.023503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce9642-bb88-45ec-b201-4af3bf47d365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:31.025295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ce9642-bb88-45ec-b201-4af3bf47d365 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:31.037227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1321/2664] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:49:31.041490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724cc4d-6438-418f-a76e-4b19f48a1785 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:31.052185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1724cc4d-6438-418f-a76e-4b19f48a1785 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:31.056937 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7b7a00-6648-4c2d-8fad-e8d3df8e1c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:31.057569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b7a00-6648-4c2d-8fad-e8d3df8e1c63 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.057849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b7a00-6648-4c2d-8fad-e8d3df8e1c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.058118 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7b7a00-6648-4c2d-8fad-e8d3df8e1c63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.058819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b7a00-6648-4c2d-8fad-e8d3df8e1c63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:31.060429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7b7a00-6648-4c2d-8fad-e8d3df8e1c63 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:31.070462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1344/2665] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:31.074970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce9642-bb88-45ec-b201-4af3bf47d365 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:31.085281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ce9642-bb88-45ec-b201-4af3bf47d365 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:31.090459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c22373-6d65-4df1-9217-5316ffbe77e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:31.091109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c22373-6d65-4df1-9217-5316ffbe77e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.091388 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c22373-6d65-4df1-9217-5316ffbe77e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.091660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c22373-6d65-4df1-9217-5316ffbe77e4 admin admin] PATH_INFO: `/v3/projects/0dccf1e9f776406ab0d37f62946b778d/users/f4fb82e220b44282a736be5c1c3de66e/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.106945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c22373-6d65-4df1-9217-5316ffbe77e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dccf1e9f776406ab0d37f62946b778d, user_id=f4fb82e220b44282a736be5c1c3de66e, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:31.109620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c22373-6d65-4df1-9217-5316ffbe77e4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:31.122415 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1322/2666] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:31 2022] PUT /identity/v3/projects/0dccf1e9f776406ab0d37f62946b778d/users/f4fb82e220b44282a736be5c1c3de66e/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:49:31.133462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8cbad-ff6c-4037-bc02-d58239fcee6f None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.133740 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8cbad-ff6c-4037-bc02-d58239fcee6f None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.134009 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8cbad-ff6c-4037-bc02-d58239fcee6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.162415 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-85f8cbad-ff6c-4037-bc02-d58239fcee6f None None] MFA Rules not processed for user `f4fb82e220b44282a736be5c1c3de66e`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:31.163615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-85f8cbad-ff6c-4037-bc02-d58239fcee6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:31.228566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1345/2667] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:31 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:31.233192 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c22373-6d65-4df1-9217-5316ffbe77e4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:31.287921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:31.301341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c22373-6d65-4df1-9217-5316ffbe77e4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:31.305916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da08ad-b4cd-4bd8-85d5-8af676144677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:31.306518 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da08ad-b4cd-4bd8-85d5-8af676144677 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.306796 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da08ad-b4cd-4bd8-85d5-8af676144677 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.307612 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1da08ad-b4cd-4bd8-85d5-8af676144677 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.308728 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da08ad-b4cd-4bd8-85d5-8af676144677 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:31.310644 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1da08ad-b4cd-4bd8-85d5-8af676144677 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:31.337058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1323/2668] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:31 2022] POST /identity/v3/projects => generated 407 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:31.341508 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8cbad-ff6c-4037-bc02-d58239fcee6f None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:31.353619 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8cbad-ff6c-4037-bc02-d58239fcee6f None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:31.358050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:31.411751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc59242-7643-4edf-ac86-dda5396cae7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:31.412352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc59242-7643-4edf-ac86-dda5396cae7b admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.412629 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc59242-7643-4edf-ac86-dda5396cae7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.412942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc59242-7643-4edf-ac86-dda5396cae7b admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.413555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc59242-7643-4edf-ac86-dda5396cae7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:31.415445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc59242-7643-4edf-ac86-dda5396cae7b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:31.437688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1346/2669] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:31 2022] POST /identity/v3/users => generated 310 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:31.441785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da08ad-b4cd-4bd8-85d5-8af676144677 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:31.452038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1da08ad-b4cd-4bd8-85d5-8af676144677 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:31.456889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bfc60-74c2-4f08-bec8-e34a4318c3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:31.457492 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bfc60-74c2-4f08-bec8-e34a4318c3fc admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.457772 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bfc60-74c2-4f08-bec8-e34a4318c3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.458045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121bfc60-74c2-4f08-bec8-e34a4318c3fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.458680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bfc60-74c2-4f08-bec8-e34a4318c3fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:31.460371 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121bfc60-74c2-4f08-bec8-e34a4318c3fc admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:31.471253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1324/2670] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:31.475311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc59242-7643-4edf-ac86-dda5396cae7b admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:31.485682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc59242-7643-4edf-ac86-dda5396cae7b admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:31.490386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638bbbc-f471-43ab-8047-0efe1ec7598e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:31.491109 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638bbbc-f471-43ab-8047-0efe1ec7598e admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.491379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638bbbc-f471-43ab-8047-0efe1ec7598e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.491642 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3638bbbc-f471-43ab-8047-0efe1ec7598e admin admin] PATH_INFO: `/v3/projects/f5873edc076d488c8a9f43a3accee9e3/users/a1201c9659fd4f76a9b460f32a004306/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.506709 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638bbbc-f471-43ab-8047-0efe1ec7598e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5873edc076d488c8a9f43a3accee9e3, user_id=a1201c9659fd4f76a9b460f32a004306, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:31.509350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3638bbbc-f471-43ab-8047-0efe1ec7598e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:31.521631 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1347/2671] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:31 2022] PUT /identity/v3/projects/f5873edc076d488c8a9f43a3accee9e3/users/a1201c9659fd4f76a9b460f32a004306/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:31.525494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bfc60-74c2-4f08-bec8-e34a4318c3fc admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:31.535479 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121bfc60-74c2-4f08-bec8-e34a4318c3fc admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:31.591665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c153b59-361f-47c4-9df8-b6a642da0eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:31.592263 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c153b59-361f-47c4-9df8-b6a642da0eb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.592536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c153b59-361f-47c4-9df8-b6a642da0eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.592828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c153b59-361f-47c4-9df8-b6a642da0eb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.593456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c153b59-361f-47c4-9df8-b6a642da0eb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:31.595026 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c153b59-361f-47c4-9df8-b6a642da0eb5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:31.605007 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1325/2672] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:31 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:31.609224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638bbbc-f471-43ab-8047-0efe1ec7598e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:31.621002 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3638bbbc-f471-43ab-8047-0efe1ec7598e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:31.625640 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd56cf6-1490-4360-a3d1-c2a8ab9f5710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:31.626289 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd56cf6-1490-4360-a3d1-c2a8ab9f5710 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.626567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd56cf6-1490-4360-a3d1-c2a8ab9f5710 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.626833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd56cf6-1490-4360-a3d1-c2a8ab9f5710 admin admin] PATH_INFO: `/v3/projects/f5873edc076d488c8a9f43a3accee9e3/users/a1201c9659fd4f76a9b460f32a004306/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.628981 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd56cf6-1490-4360-a3d1-c2a8ab9f5710 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5873edc076d488c8a9f43a3accee9e3, user_id=a1201c9659fd4f76a9b460f32a004306, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:31.631572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd56cf6-1490-4360-a3d1-c2a8ab9f5710 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:31.642417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1348/2673] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:31 2022] PUT /identity/v3/projects/f5873edc076d488c8a9f43a3accee9e3/users/a1201c9659fd4f76a9b460f32a004306/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:31.653227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521ae3b-d5b6-47b9-89cc-93b44feb2626 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.653590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521ae3b-d5b6-47b9-89cc-93b44feb2626 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.653935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521ae3b-d5b6-47b9-89cc-93b44feb2626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.683404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0521ae3b-d5b6-47b9-89cc-93b44feb2626 None None] MFA Rules not processed for user `a1201c9659fd4f76a9b460f32a004306`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:31.684741 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0521ae3b-d5b6-47b9-89cc-93b44feb2626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:31.754101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1326/2674] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:31 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:31.764834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd56cf6-1490-4360-a3d1-c2a8ab9f5710 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:31.766689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1fd56cf6-1490-4360-a3d1-c2a8ab9f5710 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:31.827168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd56cf6-1490-4360-a3d1-c2a8ab9f5710 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:31.830816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609ce41-1313-46d2-b2bc-27bdc1a35478 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:31.831410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609ce41-1313-46d2-b2bc-27bdc1a35478 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:31.831685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609ce41-1313-46d2-b2bc-27bdc1a35478 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:31.831953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8609ce41-1313-46d2-b2bc-27bdc1a35478 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:31.840667 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609ce41-1313-46d2-b2bc-27bdc1a35478 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:31.842108 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8609ce41-1313-46d2-b2bc-27bdc1a35478 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:31.851334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1349/2675] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:31 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:32.007595 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521ae3b-d5b6-47b9-89cc-93b44feb2626 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:32.017962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521ae3b-d5b6-47b9-89cc-93b44feb2626 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:32.021583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abd759e-9a52-4585-b614-9dead6828661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:32.022178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd759e-9a52-4585-b614-9dead6828661 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:32.022459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd759e-9a52-4585-b614-9dead6828661 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:32.022713 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abd759e-9a52-4585-b614-9dead6828661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:32.078304 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:32.097825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd759e-9a52-4585-b614-9dead6828661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:32.099347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abd759e-9a52-4585-b614-9dead6828661 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:32.110115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1327/2676] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:32 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:39.504264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609ce41-1313-46d2-b2bc-27bdc1a35478 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:39.581352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8609ce41-1313-46d2-b2bc-27bdc1a35478 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:39.586425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:39.587234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:39.587535 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:39.587796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] PATH_INFO: `/v3/users/f4fb82e220b44282a736be5c1c3de66e` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:39.589003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4fb82e220b44282a736be5c1c3de66e)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:39.592463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:39.598483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:39.659359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4fb82e220b44282a736be5c1c3de66e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:39.668620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4fb82e220b44282a736be5c1c3de66e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:39.669256 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4fb82e220b44282a736be5c1c3de66e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:39.681311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4fb82e220b44282a736be5c1c3de66e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:39.688091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4fb82e220b44282a736be5c1c3de66e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:39.694188 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1350/2677] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:39 2022] DELETE /identity/v3/users/f4fb82e220b44282a736be5c1c3de66e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:39.699420 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825701a-5b3d-41cb-ba8e-e0f88064b961 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:39.699903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825701a-5b3d-41cb-ba8e-e0f88064b961 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:39.700082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d825701a-5b3d-41cb-ba8e-e0f88064b961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:39.719326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-d825701a-5b3d-41cb-ba8e-e0f88064b961 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:39.720501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d825701a-5b3d-41cb-ba8e-e0f88064b961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:39.776564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1328/2678] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:39.787170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:39.788542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:39.846118 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c634ac2f-0770-48e6-92d3-18fcc15beb60 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:39.849766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc74708-84d3-4370-9693-bf611c05efe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:39.850295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc74708-84d3-4370-9693-bf611c05efe5 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:39.850565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc74708-84d3-4370-9693-bf611c05efe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:39.850831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc74708-84d3-4370-9693-bf611c05efe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:39.859511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc74708-84d3-4370-9693-bf611c05efe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:39.860942 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc74708-84d3-4370-9693-bf611c05efe5 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:39.870064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1351/2679] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:40.140485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825701a-5b3d-41cb-ba8e-e0f88064b961 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:40.142075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-d825701a-5b3d-41cb-ba8e-e0f88064b961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:40.155365 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d825701a-5b3d-41cb-ba8e-e0f88064b961 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:40.160245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:40.161583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:40.162479 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:40.162750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] PATH_INFO: `/v3/projects/0dccf1e9f776406ab0d37f62946b778d` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:40.164096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dccf1e9f776406ab0d37f62946b778d)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:40.165921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:40.213312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dccf1e9f776406ab0d37f62946b778d', 'request_id': 'req-8b1502ae-0828-49b5-a2fa-7e82927c547d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.219668 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dccf1e9f776406ab0d37f62946b778d', 'request_id': 'req-8b1502ae-0828-49b5-a2fa-7e82927c547d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.227037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dccf1e9f776406ab0d37f62946b778d', 'request_id': 'req-8b1502ae-0828-49b5-a2fa-7e82927c547d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.237198 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] The token cache is being invalidate because project 0dccf1e9f776406ab0d37f62946b778d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:40.237485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.238112 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.239797 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1329/2680] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:40 2022] DELETE /identity/v3/projects/0dccf1e9f776406ab0d37f62946b778d => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:40.243733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc74708-84d3-4370-9693-bf611c05efe5 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:40.253945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc74708-84d3-4370-9693-bf611c05efe5 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:40.312379 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e259cc-8e10-4eac-b2da-b15ad43a3903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:40.313053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e259cc-8e10-4eac-b2da-b15ad43a3903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:40.313324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e259cc-8e10-4eac-b2da-b15ad43a3903 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:40.313591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e259cc-8e10-4eac-b2da-b15ad43a3903 admin admin] PATH_INFO: `/v3/users/a1201c9659fd4f76a9b460f32a004306` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:40.314689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e259cc-8e10-4eac-b2da-b15ad43a3903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1201c9659fd4f76a9b460f32a004306)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:40.316659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1e259cc-8e10-4eac-b2da-b15ad43a3903 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:40.344431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:40.367961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e1e259cc-8e10-4eac-b2da-b15ad43a3903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1201c9659fd4f76a9b460f32a004306'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.375188 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e1e259cc-8e10-4eac-b2da-b15ad43a3903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1201c9659fd4f76a9b460f32a004306'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.375911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e1e259cc-8e10-4eac-b2da-b15ad43a3903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1201c9659fd4f76a9b460f32a004306'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.389520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e1e259cc-8e10-4eac-b2da-b15ad43a3903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1201c9659fd4f76a9b460f32a004306'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.396438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e1e259cc-8e10-4eac-b2da-b15ad43a3903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1201c9659fd4f76a9b460f32a004306'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.403069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1352/2681] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:40 2022] DELETE /identity/v3/users/a1201c9659fd4f76a9b460f32a004306 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:40.678935 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:40.680697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:40.707394 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1502ae-0828-49b5-a2fa-7e82927c547d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:40.774511 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:40.775206 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:40.775558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:40.775819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] PATH_INFO: `/v3/projects/f5873edc076d488c8a9f43a3accee9e3` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:40.777979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5873edc076d488c8a9f43a3accee9e3)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:40.779667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:40.790742 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:40.834148 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5873edc076d488c8a9f43a3accee9e3', 'request_id': 'req-c941498a-0377-42e4-8471-38a364d57490'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.843287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5873edc076d488c8a9f43a3accee9e3', 'request_id': 'req-c941498a-0377-42e4-8471-38a364d57490'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.853003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5873edc076d488c8a9f43a3accee9e3', 'request_id': 'req-c941498a-0377-42e4-8471-38a364d57490'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.865793 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] The token cache is being invalidate because project f5873edc076d488c8a9f43a3accee9e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:40.866080 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.866762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c941498a-0377-42e4-8471-38a364d57490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:40.869937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1330/2682] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:40 2022] DELETE /identity/v3/projects/f5873edc076d488c8a9f43a3accee9e3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:40.876171 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a355e8-6911-4a4a-9056-95bea7e9f26a None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:40.876642 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a355e8-6911-4a4a-9056-95bea7e9f26a None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:40.876642 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a355e8-6911-4a4a-9056-95bea7e9f26a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:40.897253 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-f9a355e8-6911-4a4a-9056-95bea7e9f26a None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:40.898627 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f9a355e8-6911-4a4a-9056-95bea7e9f26a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:40.964961 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1353/2683] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:40.973382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d970d-186f-4a0d-bb43-b63ebd1d5163 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:40.973738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d970d-186f-4a0d-bb43-b63ebd1d5163 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:40.974082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838d970d-186f-4a0d-bb43-b63ebd1d5163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:40.995063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-838d970d-186f-4a0d-bb43-b63ebd1d5163 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:40.996391 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-838d970d-186f-4a0d-bb43-b63ebd1d5163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:41.006294 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1331/2684] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:41.010495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a355e8-6911-4a4a-9056-95bea7e9f26a None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:41.021272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a355e8-6911-4a4a-9056-95bea7e9f26a None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:41.025819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99a6eb-dea7-4cc4-b296-0b4ac269f876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:41.026417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99a6eb-dea7-4cc4-b296-0b4ac269f876 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:41.026708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99a6eb-dea7-4cc4-b296-0b4ac269f876 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:41.027520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db99a6eb-dea7-4cc4-b296-0b4ac269f876 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:41.028342 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99a6eb-dea7-4cc4-b296-0b4ac269f876 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:49:41.028633 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99a6eb-dea7-4cc4-b296-0b4ac269f876 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:41.030266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db99a6eb-dea7-4cc4-b296-0b4ac269f876 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:41.041686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1354/2685] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:49:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:49:41.046445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d970d-186f-4a0d-bb43-b63ebd1d5163 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:41.055717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838d970d-186f-4a0d-bb43-b63ebd1d5163 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:41.061010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37218a-66d7-4722-b3d0-8992c4b1f91d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:41.061497 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37218a-66d7-4722-b3d0-8992c4b1f91d admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:41.061775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37218a-66d7-4722-b3d0-8992c4b1f91d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:41.062045 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a37218a-66d7-4722-b3d0-8992c4b1f91d admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:41.062681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37218a-66d7-4722-b3d0-8992c4b1f91d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:41.064570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a37218a-66d7-4722-b3d0-8992c4b1f91d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:41.096667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1332/2686] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:41 2022] POST /identity/v3/projects => generated 385 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:49:41.100637 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99a6eb-dea7-4cc4-b296-0b4ac269f876 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:41.112929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db99a6eb-dea7-4cc4-b296-0b4ac269f876 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:41.173292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433a477-d314-4c08-aaae-acf71159c591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:41.173963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433a477-d314-4c08-aaae-acf71159c591 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:41.174235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433a477-d314-4c08-aaae-acf71159c591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:41.174503 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2433a477-d314-4c08-aaae-acf71159c591 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:41.175141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433a477-d314-4c08-aaae-acf71159c591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:41.177025 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2433a477-d314-4c08-aaae-acf71159c591 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:41.199635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1355/2687] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:41 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:41.212487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37218a-66d7-4722-b3d0-8992c4b1f91d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:41.215502 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a37218a-66d7-4722-b3d0-8992c4b1f91d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:41.220487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb273501-8069-4589-b8b2-2d7dbd84d473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:41.221116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb273501-8069-4589-b8b2-2d7dbd84d473 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:41.221398 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb273501-8069-4589-b8b2-2d7dbd84d473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:41.221672 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb273501-8069-4589-b8b2-2d7dbd84d473 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:41.222344 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb273501-8069-4589-b8b2-2d7dbd84d473 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:41.223952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb273501-8069-4589-b8b2-2d7dbd84d473 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:41.234685 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1333/2688] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:41.238721 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433a477-d314-4c08-aaae-acf71159c591 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:41.249105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2433a477-d314-4c08-aaae-acf71159c591 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:41.253960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e8ed2f-fe59-4699-9173-bc2998abd324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:41.254660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8ed2f-fe59-4699-9173-bc2998abd324 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:41.254955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8ed2f-fe59-4699-9173-bc2998abd324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:41.255251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e8ed2f-fe59-4699-9173-bc2998abd324 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:41.256004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8ed2f-fe59-4699-9173-bc2998abd324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:41.257679 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e8ed2f-fe59-4699-9173-bc2998abd324 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:41.272383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1356/2689] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:41.273684 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb273501-8069-4589-b8b2-2d7dbd84d473 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:41.285291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb273501-8069-4589-b8b2-2d7dbd84d473 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:41.290611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfb371-b542-415b-ac71-9dbe4eb662ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:41.291105 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfb371-b542-415b-ac71-9dbe4eb662ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:41.291385 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfb371-b542-415b-ac71-9dbe4eb662ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:41.291658 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfb371-b542-415b-ac71-9dbe4eb662ea admin admin] PATH_INFO: `/v3/projects/c8e396d242564d12b8e404ffdb2368cd/users/ae9cec6885ee45b4824a1df9c9748ec9/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:41.313843 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfb371-b542-415b-ac71-9dbe4eb662ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e396d242564d12b8e404ffdb2368cd, user_id=ae9cec6885ee45b4824a1df9c9748ec9, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:41.316494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bfb371-b542-415b-ac71-9dbe4eb662ea admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:41.330048 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1334/2690] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:41 2022] PUT /identity/v3/projects/c8e396d242564d12b8e404ffdb2368cd/users/ae9cec6885ee45b4824a1df9c9748ec9/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:41.341686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb84f68-304f-4638-a0da-ed8ef931ef53 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:41.341973 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb84f68-304f-4638-a0da-ed8ef931ef53 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:41.342437 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb84f68-304f-4638-a0da-ed8ef931ef53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:41.364570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:41.378616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-1fb84f68-304f-4638-a0da-ed8ef931ef53 None None] MFA Rules not processed for user `ae9cec6885ee45b4824a1df9c9748ec9`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:41.380831 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1fb84f68-304f-4638-a0da-ed8ef931ef53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:41.446106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1357/2691] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:41.458552 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfb371-b542-415b-ac71-9dbe4eb662ea admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:41.460407 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-59bfb371-b542-415b-ac71-9dbe4eb662ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:41.523353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfb371-b542-415b-ac71-9dbe4eb662ea admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:41.527578 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3e49dc-1a45-4ccc-9f26-3a57d8b15af0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:41.528180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e49dc-1a45-4ccc-9f26-3a57d8b15af0 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:41.528453 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e49dc-1a45-4ccc-9f26-3a57d8b15af0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:41.528720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3e49dc-1a45-4ccc-9f26-3a57d8b15af0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:41.536932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e49dc-1a45-4ccc-9f26-3a57d8b15af0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:41.538336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3e49dc-1a45-4ccc-9f26-3a57d8b15af0 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:41.547547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1335/2692] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:41.567512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb84f68-304f-4638-a0da-ed8ef931ef53 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:41.629087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb84f68-304f-4638-a0da-ed8ef931ef53 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:41.633810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:41.634470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:41.634819 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:41.635222 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] PATH_INFO: `/v3/users/ae9cec6885ee45b4824a1df9c9748ec9` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:41.637033 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae9cec6885ee45b4824a1df9c9748ec9)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:41.639083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:41.686313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae9cec6885ee45b4824a1df9c9748ec9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:41.693680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae9cec6885ee45b4824a1df9c9748ec9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:41.694307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae9cec6885ee45b4824a1df9c9748ec9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:41.707067 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae9cec6885ee45b4824a1df9c9748ec9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:41.714031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae9cec6885ee45b4824a1df9c9748ec9'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:41.719946 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1358/2693] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:41 2022] DELETE /identity/v3/users/ae9cec6885ee45b4824a1df9c9748ec9 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:41.724366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df105b24-8141-4ab1-bec0-d379beb9a87d None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:41.724861 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df105b24-8141-4ab1-bec0-d379beb9a87d None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:41.724963 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df105b24-8141-4ab1-bec0-d379beb9a87d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:41.743959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-df105b24-8141-4ab1-bec0-d379beb9a87d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:41.745166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-df105b24-8141-4ab1-bec0-d379beb9a87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:41.779990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:41.811082 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1336/2694] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:41.820652 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:41.822058 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:41.842248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:41.880918 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b281a9f-e3f3-4724-9e85-883cf5b0bc7b admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:41.884648 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50715822-7b95-4b58-b700-7f9481216452 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:41.885269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50715822-7b95-4b58-b700-7f9481216452 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:41.885542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50715822-7b95-4b58-b700-7f9481216452 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:41.885807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50715822-7b95-4b58-b700-7f9481216452 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:41.894874 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50715822-7b95-4b58-b700-7f9481216452 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:41.896277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50715822-7b95-4b58-b700-7f9481216452 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:41.906470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1359/2695] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:42.179180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df105b24-8141-4ab1-bec0-d379beb9a87d None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:42.180644 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-df105b24-8141-4ab1-bec0-d379beb9a87d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:42.196277 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df105b24-8141-4ab1-bec0-d379beb9a87d None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:42.202477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:42.203114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.203426 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.203644 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] PATH_INFO: `/v3/projects/c8e396d242564d12b8e404ffdb2368cd` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.204894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e396d242564d12b8e404ffdb2368cd)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:42.206857 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:42.253449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e396d242564d12b8e404ffdb2368cd', 'request_id': 'req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:42.259409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e396d242564d12b8e404ffdb2368cd', 'request_id': 'req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:42.267862 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e396d242564d12b8e404ffdb2368cd', 'request_id': 'req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:42.278687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] The token cache is being invalidate because project c8e396d242564d12b8e404ffdb2368cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:42.278966 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:42.279571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99ac57b1-7aa9-42e0-8ecc-26828ea4722d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:42.281084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1337/2696] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:42 2022] DELETE /identity/v3/projects/c8e396d242564d12b8e404ffdb2368cd => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:42.285947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b45cb7-3128-4c13-a848-16b3f72df6d4 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.286230 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b45cb7-3128-4c13-a848-16b3f72df6d4 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.286501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b45cb7-3128-4c13-a848-16b3f72df6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.305558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-f2b45cb7-3128-4c13-a848-16b3f72df6d4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:42.306739 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f2b45cb7-3128-4c13-a848-16b3f72df6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:42.364013 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1360/2697] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:42.372867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23228db7-15ef-4e69-b353-fe24a664dcf0 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.373150 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23228db7-15ef-4e69-b353-fe24a664dcf0 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.373418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23228db7-15ef-4e69-b353-fe24a664dcf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.392980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-23228db7-15ef-4e69-b353-fe24a664dcf0 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:42.394152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-23228db7-15ef-4e69-b353-fe24a664dcf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:42.402455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1338/2698] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:42.406825 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b45cb7-3128-4c13-a848-16b3f72df6d4 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:42.416425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b45cb7-3128-4c13-a848-16b3f72df6d4 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:42.422108 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea720e5-ff8a-4622-83a0-c6294ca8ad94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:42.423191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea720e5-ff8a-4622-83a0-c6294ca8ad94 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.423489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea720e5-ff8a-4622-83a0-c6294ca8ad94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.423724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea720e5-ff8a-4622-83a0-c6294ca8ad94 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.424398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea720e5-ff8a-4622-83a0-c6294ca8ad94 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:49:42.424670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea720e5-ff8a-4622-83a0-c6294ca8ad94 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:42.426331 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea720e5-ff8a-4622-83a0-c6294ca8ad94 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:42.439412 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1361/2699] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:49:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:49:42.445484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23228db7-15ef-4e69-b353-fe24a664dcf0 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:42.455664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23228db7-15ef-4e69-b353-fe24a664dcf0 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:42.460214 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32659ee8-1220-45a3-972f-bac6c9886525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:42.460831 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32659ee8-1220-45a3-972f-bac6c9886525 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.461103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32659ee8-1220-45a3-972f-bac6c9886525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.461367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32659ee8-1220-45a3-972f-bac6c9886525 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.461979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32659ee8-1220-45a3-972f-bac6c9886525 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:42.463865 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32659ee8-1220-45a3-972f-bac6c9886525 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:42.490395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1339/2700] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:42 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:49:42.494579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea720e5-ff8a-4622-83a0-c6294ca8ad94 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:42.506737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea720e5-ff8a-4622-83a0-c6294ca8ad94 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:42.563313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a3122-7847-49cf-a06a-5a132d2bc3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:42.563911 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a3122-7847-49cf-a06a-5a132d2bc3b3 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.564184 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a3122-7847-49cf-a06a-5a132d2bc3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.564451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617a3122-7847-49cf-a06a-5a132d2bc3b3 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.565406 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a3122-7847-49cf-a06a-5a132d2bc3b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:42.567287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617a3122-7847-49cf-a06a-5a132d2bc3b3 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:42.588855 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1362/2701] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:42 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:49:42.593014 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32659ee8-1220-45a3-972f-bac6c9886525 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:42.603136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32659ee8-1220-45a3-972f-bac6c9886525 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:42.608187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce7b18-e0da-458e-a126-7f67853f1b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:42.608794 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce7b18-e0da-458e-a126-7f67853f1b2a admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.609069 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce7b18-e0da-458e-a126-7f67853f1b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.609338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bce7b18-e0da-458e-a126-7f67853f1b2a admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.609961 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce7b18-e0da-458e-a126-7f67853f1b2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:42.611535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bce7b18-e0da-458e-a126-7f67853f1b2a admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:42.621760 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1340/2702] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:42.625507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a3122-7847-49cf-a06a-5a132d2bc3b3 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:42.635239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617a3122-7847-49cf-a06a-5a132d2bc3b3 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:42.639999 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f39637-c104-45b6-b79a-afcdd6b37517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:42.640588 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f39637-c104-45b6-b79a-afcdd6b37517 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.640904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f39637-c104-45b6-b79a-afcdd6b37517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.641175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f39637-c104-45b6-b79a-afcdd6b37517 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.641805 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f39637-c104-45b6-b79a-afcdd6b37517 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:42.643353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f39637-c104-45b6-b79a-afcdd6b37517 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:42.654799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1363/2703] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:42.658750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce7b18-e0da-458e-a126-7f67853f1b2a admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:42.669231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bce7b18-e0da-458e-a126-7f67853f1b2a admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:42.673883 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11889399-aefc-412a-9a62-001b41553c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:42.674525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11889399-aefc-412a-9a62-001b41553c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.674803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11889399-aefc-412a-9a62-001b41553c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.675137 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11889399-aefc-412a-9a62-001b41553c04 admin admin] PATH_INFO: `/v3/projects/03271095a8554c9db933017c03566f7c/users/5539595af9284a87ab2a88783cb9dc12/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.690097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11889399-aefc-412a-9a62-001b41553c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03271095a8554c9db933017c03566f7c, user_id=5539595af9284a87ab2a88783cb9dc12, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:42.692717 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11889399-aefc-412a-9a62-001b41553c04 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:42.705500 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1341/2704] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:42 2022] PUT /identity/v3/projects/03271095a8554c9db933017c03566f7c/users/5539595af9284a87ab2a88783cb9dc12/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:42.716086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de9ebd-8e87-4a62-9f24-94b8293714b0 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.716389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de9ebd-8e87-4a62-9f24-94b8293714b0 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.716660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de9ebd-8e87-4a62-9f24-94b8293714b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.744440 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-b1de9ebd-8e87-4a62-9f24-94b8293714b0 None None] MFA Rules not processed for user `5539595af9284a87ab2a88783cb9dc12`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:42.745870 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b1de9ebd-8e87-4a62-9f24-94b8293714b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:42.758113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:42.810243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1364/2705] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:42 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:49:42.814786 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11889399-aefc-412a-9a62-001b41553c04 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:42.871824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:42.875649 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11889399-aefc-412a-9a62-001b41553c04 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:42.881649 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cb2a0-82f1-47b5-8a50-f663600056ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:42.882467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cb2a0-82f1-47b5-8a50-f663600056ca admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.882791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cb2a0-82f1-47b5-8a50-f663600056ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.883235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93cb2a0-82f1-47b5-8a50-f663600056ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.883979 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93cb2a0-82f1-47b5-8a50-f663600056ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:42.886256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93cb2a0-82f1-47b5-8a50-f663600056ca admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:42.911973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1342/2706] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:42 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:42.916073 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de9ebd-8e87-4a62-9f24-94b8293714b0 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:42.927337 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de9ebd-8e87-4a62-9f24-94b8293714b0 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:42.982816 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdda46-1112-4f7d-99b4-3d0fc82a2518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:42.983501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdda46-1112-4f7d-99b4-3d0fc82a2518 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:42.983775 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdda46-1112-4f7d-99b4-3d0fc82a2518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:42.984043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cdda46-1112-4f7d-99b4-3d0fc82a2518 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:42.984662 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdda46-1112-4f7d-99b4-3d0fc82a2518 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:42.986591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cdda46-1112-4f7d-99b4-3d0fc82a2518 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:43.007292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1365/2707] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:42 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:43.011397 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cb2a0-82f1-47b5-8a50-f663600056ca admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:43.021388 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93cb2a0-82f1-47b5-8a50-f663600056ca admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:43.025925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd52b19-f675-4231-9af1-560f6106ede9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:43.026507 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd52b19-f675-4231-9af1-560f6106ede9 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.027494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd52b19-f675-4231-9af1-560f6106ede9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.027766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd52b19-f675-4231-9af1-560f6106ede9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.028397 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd52b19-f675-4231-9af1-560f6106ede9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:43.029998 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd52b19-f675-4231-9af1-560f6106ede9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:43.040240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1343/2708] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:43.044705 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdda46-1112-4f7d-99b4-3d0fc82a2518 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:43.054608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cdda46-1112-4f7d-99b4-3d0fc82a2518 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:43.059259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc76020-a698-4545-b0f7-a41ccd03df5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:43.059849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc76020-a698-4545-b0f7-a41ccd03df5c admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.060128 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc76020-a698-4545-b0f7-a41ccd03df5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.060399 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc76020-a698-4545-b0f7-a41ccd03df5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.061074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc76020-a698-4545-b0f7-a41ccd03df5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:43.062630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc76020-a698-4545-b0f7-a41ccd03df5c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:43.072634 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1366/2709] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:43.077114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd52b19-f675-4231-9af1-560f6106ede9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:43.087078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd52b19-f675-4231-9af1-560f6106ede9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:43.092142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b934c29-0d3f-4370-9593-d7d9307591ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:43.092804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b934c29-0d3f-4370-9593-d7d9307591ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.093073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b934c29-0d3f-4370-9593-d7d9307591ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.093341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b934c29-0d3f-4370-9593-d7d9307591ec admin admin] PATH_INFO: `/v3/projects/12e6a55ed65049cfa2924f56e4611d40/users/85aeba8d18824e58b6cd05f1fb0913b0/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.108002 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b934c29-0d3f-4370-9593-d7d9307591ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=12e6a55ed65049cfa2924f56e4611d40, user_id=85aeba8d18824e58b6cd05f1fb0913b0, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:43.110656 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b934c29-0d3f-4370-9593-d7d9307591ec admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:43.122907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1344/2710] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:43 2022] PUT /identity/v3/projects/12e6a55ed65049cfa2924f56e4611d40/users/85aeba8d18824e58b6cd05f1fb0913b0/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:43.188251 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2bb75-6025-4801-9078-e0394998f633 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.188536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2bb75-6025-4801-9078-e0394998f633 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.188835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2bb75-6025-4801-9078-e0394998f633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.217387 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e1d2bb75-6025-4801-9078-e0394998f633 None None] MFA Rules not processed for user `85aeba8d18824e58b6cd05f1fb0913b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:43.218570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e1d2bb75-6025-4801-9078-e0394998f633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:43.281414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1367/2711] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:43 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:43.285975 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b934c29-0d3f-4370-9593-d7d9307591ec admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:43.351063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b934c29-0d3f-4370-9593-d7d9307591ec admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:43.356983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58cb3e-fc67-4619-aaff-54774b510d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:43.357591 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58cb3e-fc67-4619-aaff-54774b510d62 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.357869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58cb3e-fc67-4619-aaff-54774b510d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.358140 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc58cb3e-fc67-4619-aaff-54774b510d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.358768 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58cb3e-fc67-4619-aaff-54774b510d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:43.360660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc58cb3e-fc67-4619-aaff-54774b510d62 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:43.388430 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1345/2712] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:49:43 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:43.392521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2bb75-6025-4801-9078-e0394998f633 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:43.403507 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2bb75-6025-4801-9078-e0394998f633 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:43.422938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:43.459367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0306e8-e2d9-41cf-a6ac-2061860ada1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:43.459968 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0306e8-e2d9-41cf-a6ac-2061860ada1a admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.460243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0306e8-e2d9-41cf-a6ac-2061860ada1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.460514 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0306e8-e2d9-41cf-a6ac-2061860ada1a admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.461254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0306e8-e2d9-41cf-a6ac-2061860ada1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:43.463130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0306e8-e2d9-41cf-a6ac-2061860ada1a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:43.484318 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1368/2713] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:49:43 2022] POST /identity/v3/users => generated 298 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:49:43.488321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58cb3e-fc67-4619-aaff-54774b510d62 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:43.498601 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc58cb3e-fc67-4619-aaff-54774b510d62 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:43.503142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59112563-3418-4065-be06-60dcbd84eaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:43.503727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59112563-3418-4065-be06-60dcbd84eaaf admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.503999 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59112563-3418-4065-be06-60dcbd84eaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.504267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59112563-3418-4065-be06-60dcbd84eaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.504925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59112563-3418-4065-be06-60dcbd84eaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:43.506494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59112563-3418-4065-be06-60dcbd84eaaf admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:43.517462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1346/2714] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:43.521375 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0306e8-e2d9-41cf-a6ac-2061860ada1a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:43.531927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0306e8-e2d9-41cf-a6ac-2061860ada1a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:43.536345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3764bc5-89eb-4fe9-81d7-b93b385a6f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:43.537009 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3764bc5-89eb-4fe9-81d7-b93b385a6f6c admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.537285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3764bc5-89eb-4fe9-81d7-b93b385a6f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.537554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3764bc5-89eb-4fe9-81d7-b93b385a6f6c admin admin] PATH_INFO: `/v3/projects/7a85aa318f0140b5bf570bd44e424a62/users/9b657eb6922243448626dd6e968ab4a0/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.557271 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3764bc5-89eb-4fe9-81d7-b93b385a6f6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a85aa318f0140b5bf570bd44e424a62, user_id=9b657eb6922243448626dd6e968ab4a0, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:43.560228 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3764bc5-89eb-4fe9-81d7-b93b385a6f6c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:43.573357 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1369/2715] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:43 2022] PUT /identity/v3/projects/7a85aa318f0140b5bf570bd44e424a62/users/9b657eb6922243448626dd6e968ab4a0/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:43.578102 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59112563-3418-4065-be06-60dcbd84eaaf admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:43.588266 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59112563-3418-4065-be06-60dcbd84eaaf admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:43.643343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e9e4cc-2230-4d0f-adec-40e1aeb2ec39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:43.643944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9e4cc-2230-4d0f-adec-40e1aeb2ec39 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.644218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9e4cc-2230-4d0f-adec-40e1aeb2ec39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.644484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e9e4cc-2230-4d0f-adec-40e1aeb2ec39 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.645144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9e4cc-2230-4d0f-adec-40e1aeb2ec39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:43.646721 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e9e4cc-2230-4d0f-adec-40e1aeb2ec39 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:43.656693 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1347/2716] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:49:43 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:49:43.660891 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3764bc5-89eb-4fe9-81d7-b93b385a6f6c admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:43.671315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3764bc5-89eb-4fe9-81d7-b93b385a6f6c admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:43.675956 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db9345-a0ef-4c7f-a330-6c2887ff7898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:43.676909 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db9345-a0ef-4c7f-a330-6c2887ff7898 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.677193 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db9345-a0ef-4c7f-a330-6c2887ff7898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.677467 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db9345-a0ef-4c7f-a330-6c2887ff7898 admin admin] PATH_INFO: `/v3/projects/7a85aa318f0140b5bf570bd44e424a62/users/9b657eb6922243448626dd6e968ab4a0/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.679442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db9345-a0ef-4c7f-a330-6c2887ff7898 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a85aa318f0140b5bf570bd44e424a62, user_id=9b657eb6922243448626dd6e968ab4a0, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:43.682116 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db9345-a0ef-4c7f-a330-6c2887ff7898 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:43.692213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1370/2717] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:49:43 2022] PUT /identity/v3/projects/7a85aa318f0140b5bf570bd44e424a62/users/9b657eb6922243448626dd6e968ab4a0/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:43.702940 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a682714-3c29-4018-a39d-be1ae37f0021 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.703221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a682714-3c29-4018-a39d-be1ae37f0021 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.703491 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a682714-3c29-4018-a39d-be1ae37f0021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.730490 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0a682714-3c29-4018-a39d-be1ae37f0021 None None] MFA Rules not processed for user `9b657eb6922243448626dd6e968ab4a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:43.731686 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0a682714-3c29-4018-a39d-be1ae37f0021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:43.736607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:43.801387 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1348/2718] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:43 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:43.813197 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db9345-a0ef-4c7f-a330-6c2887ff7898 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:43.814922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-55db9345-a0ef-4c7f-a330-6c2887ff7898 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:43.871055 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db9345-a0ef-4c7f-a330-6c2887ff7898 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:43.874955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce2a75-ef11-432e-8709-70c47b346b8d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:43.875540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce2a75-ef11-432e-8709-70c47b346b8d service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:43.875812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce2a75-ef11-432e-8709-70c47b346b8d service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:43.876078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ce2a75-ef11-432e-8709-70c47b346b8d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:43.929343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce2a75-ef11-432e-8709-70c47b346b8d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:43.930747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ce2a75-ef11-432e-8709-70c47b346b8d service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:43.939815 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1371/2719] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:45.455124 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f90f41-5889-4b30-b961-8231856be905 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:45.455580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f90f41-5889-4b30-b961-8231856be905 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:45.455760 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f90f41-5889-4b30-b961-8231856be905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:45.460550 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce2a75-ef11-432e-8709-70c47b346b8d service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:45.462214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-09ce2a75-ef11-432e-8709-70c47b346b8d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:45.474782 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ce2a75-ef11-432e-8709-70c47b346b8d service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:45.476169 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-89f90f41-5889-4b30-b961-8231856be905 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:45.477576 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-89f90f41-5889-4b30-b961-8231856be905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:45.478594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a640b-e131-4f58-b97b-6fe17a44e823 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:45.479187 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a640b-e131-4f58-b97b-6fe17a44e823 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:45.479464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a640b-e131-4f58-b97b-6fe17a44e823 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:45.479804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041a640b-e131-4f58-b97b-6fe17a44e823 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:45.511053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:45.535473 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1349/2720] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:49:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:45.544131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a640b-e131-4f58-b97b-6fe17a44e823 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:45.545582 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041a640b-e131-4f58-b97b-6fe17a44e823 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:45.548437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f90f41-5889-4b30-b961-8231856be905 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:45.550104 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-89f90f41-5889-4b30-b961-8231856be905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:45.555158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1372/2721] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:49:45.654024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f90f41-5889-4b30-b961-8231856be905 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:45.654024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5c64a-2e6a-4b93-a006-febbdad6a13c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:45.654024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5c64a-2e6a-4b93-a006-febbdad6a13c service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:45.654024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5c64a-2e6a-4b93-a006-febbdad6a13c service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:45.654496 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c5c64a-2e6a-4b93-a006-febbdad6a13c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:45.662649 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5c64a-2e6a-4b93-a006-febbdad6a13c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:45.664226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c5c64a-2e6a-4b93-a006-febbdad6a13c service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:45.674250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1350/2722] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:49:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 06 22:49:46.225154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a640b-e131-4f58-b97b-6fe17a44e823 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:46.237787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041a640b-e131-4f58-b97b-6fe17a44e823 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:46.241933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f20494-b1ff-4e0f-98c7-8124961f1866 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:46.242676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f20494-b1ff-4e0f-98c7-8124961f1866 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:46.242957 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f20494-b1ff-4e0f-98c7-8124961f1866 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:46.243218 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f20494-b1ff-4e0f-98c7-8124961f1866 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:46.253291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f20494-b1ff-4e0f-98c7-8124961f1866 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:46.254982 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f20494-b1ff-4e0f-98c7-8124961f1866 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:46.266320 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1373/2723] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:46 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:50.380582 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5c64a-2e6a-4b93-a006-febbdad6a13c service swift] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:50.424269 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=100 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:50.453116 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c5c64a-2e6a-4b93-a006-febbdad6a13c service swift] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:50.458915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:50.459918 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:50.460193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:50.460462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] PATH_INFO: `/v3/users/5539595af9284a87ab2a88783cb9dc12` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:50.461609 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5539595af9284a87ab2a88783cb9dc12)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:50.463517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:50.524141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5539595af9284a87ab2a88783cb9dc12'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:50.524909 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5539595af9284a87ab2a88783cb9dc12'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:50.537450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5539595af9284a87ab2a88783cb9dc12'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:50.550923 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5539595af9284a87ab2a88783cb9dc12'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:50.568148 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5539595af9284a87ab2a88783cb9dc12'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:50.584640 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1351/2724] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:50 2022] DELETE /identity/v3/users/5539595af9284a87ab2a88783cb9dc12 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:50.589166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bbf2d-b5fa-4bca-b751-25bb3598ae11 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:50.590530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bbf2d-b5fa-4bca-b751-25bb3598ae11 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:50.590966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502bbf2d-b5fa-4bca-b751-25bb3598ae11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:50.611978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-502bbf2d-b5fa-4bca-b751-25bb3598ae11 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:49:50.613212 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-502bbf2d-b5fa-4bca-b751-25bb3598ae11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:50.688213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1374/2725] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:49:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:49:50.695341 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:50.697089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:50.786238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a0895c-052a-4757-affb-26a8947b5913 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:50.789976 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbceea-161c-4f1f-910c-64238e3657f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:50.790571 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbceea-161c-4f1f-910c-64238e3657f5 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:50.790838 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbceea-161c-4f1f-910c-64238e3657f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:50.791092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbceea-161c-4f1f-910c-64238e3657f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:50.801889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbceea-161c-4f1f-910c-64238e3657f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:50.803299 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cbceea-161c-4f1f-910c-64238e3657f5 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:50.814138 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1352/2726] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:49:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:49:51.148487 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bbf2d-b5fa-4bca-b751-25bb3598ae11 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:51.150177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-502bbf2d-b5fa-4bca-b751-25bb3598ae11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:51.164324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502bbf2d-b5fa-4bca-b751-25bb3598ae11 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:51.170307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:51.170975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:51.171277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:51.171549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] PATH_INFO: `/v3/projects/03271095a8554c9db933017c03566f7c` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:51.173048 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=03271095a8554c9db933017c03566f7c)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:51.175199 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:51.250130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03271095a8554c9db933017c03566f7c', 'request_id': 'req-0a544a69-51e1-4e29-ba81-98c766641f3e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:51.297396 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03271095a8554c9db933017c03566f7c', 'request_id': 'req-0a544a69-51e1-4e29-ba81-98c766641f3e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:51.303686 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03271095a8554c9db933017c03566f7c', 'request_id': 'req-0a544a69-51e1-4e29-ba81-98c766641f3e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:51.315515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] The token cache is being invalidate because project 03271095a8554c9db933017c03566f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:51.315797 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:51.316430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:51.317919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1375/2727] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:51 2022] DELETE /identity/v3/projects/03271095a8554c9db933017c03566f7c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:51.322088 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbceea-161c-4f1f-910c-64238e3657f5 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:51.334944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cbceea-161c-4f1f-910c-64238e3657f5 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:51.397648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:51.398393 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:51.398745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:51.399087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] PATH_INFO: `/v3/users/85aeba8d18824e58b6cd05f1fb0913b0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:51.400297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85aeba8d18824e58b6cd05f1fb0913b0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:51.402613 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:51.405293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:51.465909 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85aeba8d18824e58b6cd05f1fb0913b0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:51.466795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85aeba8d18824e58b6cd05f1fb0913b0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:51.476460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85aeba8d18824e58b6cd05f1fb0913b0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:51.484352 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85aeba8d18824e58b6cd05f1fb0913b0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:51.508143 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85aeba8d18824e58b6cd05f1fb0913b0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:51.517645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1353/2728] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:51 2022] DELETE /identity/v3/users/85aeba8d18824e58b6cd05f1fb0913b0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:51.923574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:51.925560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:51.943117 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a544a69-51e1-4e29-ba81-98c766641f3e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:52.006724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:52.030464 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:52.031246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:52.031618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:52.031985 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] PATH_INFO: `/v3/projects/12e6a55ed65049cfa2924f56e4611d40` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:52.033245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12e6a55ed65049cfa2924f56e4611d40)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:52.035984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:52.098743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12e6a55ed65049cfa2924f56e4611d40', 'request_id': 'req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.109521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12e6a55ed65049cfa2924f56e4611d40', 'request_id': 'req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.118802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12e6a55ed65049cfa2924f56e4611d40', 'request_id': 'req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.131903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] The token cache is being invalidate because project 12e6a55ed65049cfa2924f56e4611d40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:52.132301 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.133038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.134671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1376/2729] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:51 2022] DELETE /identity/v3/projects/12e6a55ed65049cfa2924f56e4611d40 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:52.138985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:52.140544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:52.173187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e8ea54-0b85-49a9-ae7e-d7d0178268a5 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:52.255705 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfc2701-35e8-4647-bec8-1d7ab5e02163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:52.257731 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc2701-35e8-4647-bec8-1d7ab5e02163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:52.258278 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc2701-35e8-4647-bec8-1d7ab5e02163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:52.259089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfc2701-35e8-4647-bec8-1d7ab5e02163 admin admin] PATH_INFO: `/v3/users/9b657eb6922243448626dd6e968ab4a0` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:52.260804 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc2701-35e8-4647-bec8-1d7ab5e02163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b657eb6922243448626dd6e968ab4a0)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:52.263639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfc2701-35e8-4647-bec8-1d7ab5e02163 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:52.348995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5bfc2701-35e8-4647-bec8-1d7ab5e02163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b657eb6922243448626dd6e968ab4a0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.350997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5bfc2701-35e8-4647-bec8-1d7ab5e02163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b657eb6922243448626dd6e968ab4a0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.365113 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5bfc2701-35e8-4647-bec8-1d7ab5e02163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b657eb6922243448626dd6e968ab4a0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.378008 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5bfc2701-35e8-4647-bec8-1d7ab5e02163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b657eb6922243448626dd6e968ab4a0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.396618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-5bfc2701-35e8-4647-bec8-1d7ab5e02163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b657eb6922243448626dd6e968ab4a0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.398487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:52.405803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1354/2730] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:49:52 2022] DELETE /identity/v3/users/9b657eb6922243448626dd6e968ab4a0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:49:52.789905 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:49:52.791726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:49:52.824542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e52bbea-f6b0-4a7e-afa9-4dfab800efb1 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:49:52.897769 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:49:52.898717 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:49:52.899173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:49:52.899544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] PATH_INFO: `/v3/projects/7a85aa318f0140b5bf570bd44e424a62` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:49:52.900864 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a85aa318f0140b5bf570bd44e424a62)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:49:52.904263 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:49:52.960399 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a85aa318f0140b5bf570bd44e424a62', 'request_id': 'req-18d8a044-bc13-44dc-9710-5ce7679a8190'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.970408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a85aa318f0140b5bf570bd44e424a62', 'request_id': 'req-18d8a044-bc13-44dc-9710-5ce7679a8190'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.976865 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a85aa318f0140b5bf570bd44e424a62', 'request_id': 'req-18d8a044-bc13-44dc-9710-5ce7679a8190'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.988182 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:49:52.994022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] The token cache is being invalidate because project 7a85aa318f0140b5bf570bd44e424a62 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:49:52.994279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:52.994993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:49:53.002053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1377/2731] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:49:52 2022] DELETE /identity/v3/projects/7a85aa318f0140b5bf570bd44e424a62 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:50:01.077095 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391d90-a7f7-4e02-9869-919b3e81e9e7 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:01.077583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391d90-a7f7-4e02-9869-919b3e81e9e7 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:01.077698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f391d90-a7f7-4e02-9869-919b3e81e9e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:01.097775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4f391d90-a7f7-4e02-9869-919b3e81e9e7 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:50:01.098962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4f391d90-a7f7-4e02-9869-919b3e81e9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:01.157952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1355/2732] 10.222.0.39 () {58 vars in 967 bytes} [Tue Sep 6 22:50:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:50:01.170223 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:01.171581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:01.239019 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d8a044-bc13-44dc-9710-5ce7679a8190 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:01.243812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fea1a-03cf-4ea6-a3c2-d985780d12fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:01.244403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fea1a-03cf-4ea6-a3c2-d985780d12fe service swift] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:01.244683 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fea1a-03cf-4ea6-a3c2-d985780d12fe service swift] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:01.244974 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1fea1a-03cf-4ea6-a3c2-d985780d12fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:01.253309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fea1a-03cf-4ea6-a3c2-d985780d12fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:01.254851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1fea1a-03cf-4ea6-a3c2-d985780d12fe service swift] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:01.263493 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1378/2733] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:50:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:50:03.099640 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f391d90-a7f7-4e02-9869-919b3e81e9e7 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:03.101097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4f391d90-a7f7-4e02-9869-919b3e81e9e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:03.166780 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fea1a-03cf-4ea6-a3c2-d985780d12fe service swift] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:03.170082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7d1fea1a-03cf-4ea6-a3c2-d985780d12fe service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:03.176864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f391d90-a7f7-4e02-9869-919b3e81e9e7 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:03.182186 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86435bd6-59c0-4609-9506-4258e2f4131e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:03.182821 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86435bd6-59c0-4609-9506-4258e2f4131e service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:03.183184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86435bd6-59c0-4609-9506-4258e2f4131e service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:03.183989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86435bd6-59c0-4609-9506-4258e2f4131e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:03.185871 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-86435bd6-59c0-4609-9506-4258e2f4131e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:03.197422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86435bd6-59c0-4609-9506-4258e2f4131e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:03.199421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86435bd6-59c0-4609-9506-4258e2f4131e service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:03.209213 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1356/2734] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:50:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:50:03.238359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1fea1a-03cf-4ea6-a3c2-d985780d12fe service swift] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:03.242940 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c877cf-bd2e-405a-88d2-1d8310616a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:03.243555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c877cf-bd2e-405a-88d2-1d8310616a25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:03.243861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c877cf-bd2e-405a-88d2-1d8310616a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:03.244135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c877cf-bd2e-405a-88d2-1d8310616a25 admin admin] PATH_INFO: `/v3/users/e5593dc049144447859a48e458717295` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:03.245283 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c877cf-bd2e-405a-88d2-1d8310616a25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5593dc049144447859a48e458717295)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:03.248001 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c877cf-bd2e-405a-88d2-1d8310616a25 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:03.294089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-a6c877cf-bd2e-405a-88d2-1d8310616a25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5593dc049144447859a48e458717295'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:03.301556 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-a6c877cf-bd2e-405a-88d2-1d8310616a25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5593dc049144447859a48e458717295'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:03.302168 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-a6c877cf-bd2e-405a-88d2-1d8310616a25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5593dc049144447859a48e458717295'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:03.315282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-a6c877cf-bd2e-405a-88d2-1d8310616a25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5593dc049144447859a48e458717295'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:03.321932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-a6c877cf-bd2e-405a-88d2-1d8310616a25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5593dc049144447859a48e458717295'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:03.328807 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1379/2735] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:50:03 2022] DELETE /identity/v3/users/e5593dc049144447859a48e458717295 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:50:03.510830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86435bd6-59c0-4609-9506-4258e2f4131e service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:03.524718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86435bd6-59c0-4609-9506-4258e2f4131e service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:03.586158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:03.586785 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:03.587056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:03.587321 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] PATH_INFO: `/v3/projects/9b3e57c72cec4c128a09749da5a90a5d` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:03.588417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b3e57c72cec4c128a09749da5a90a5d)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:03.590584 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:03.627613 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:50:03.638328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b3e57c72cec4c128a09749da5a90a5d', 'request_id': 'req-e62b3414-de67-46c7-9575-b94f00896af9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:03.644289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b3e57c72cec4c128a09749da5a90a5d', 'request_id': 'req-e62b3414-de67-46c7-9575-b94f00896af9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:03.651573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b3e57c72cec4c128a09749da5a90a5d', 'request_id': 'req-e62b3414-de67-46c7-9575-b94f00896af9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:03.662703 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] The token cache is being invalidate because project 9b3e57c72cec4c128a09749da5a90a5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:50:03.662983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:03.663588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-e62b3414-de67-46c7-9575-b94f00896af9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:03.665097 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1357/2736] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:50:03 2022] DELETE /identity/v3/projects/9b3e57c72cec4c128a09749da5a90a5d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:50:03.669930 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9d365-74c2-4c4e-b461-3635f93c42c9 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:03.670211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9d365-74c2-4c4e-b461-3635f93c42c9 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:03.670482 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9d365-74c2-4c4e-b461-3635f93c42c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:03.689994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-6cf9d365-74c2-4c4e-b461-3635f93c42c9 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:50:03.691185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6cf9d365-74c2-4c4e-b461-3635f93c42c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:03.695328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:50:03.751340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1380/2737] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:50:03.758949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d51a3-41ac-4bf5-a867-829b21a5be28 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:03.759230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d51a3-41ac-4bf5-a867-829b21a5be28 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:03.759501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d51a3-41ac-4bf5-a867-829b21a5be28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:03.780436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c90d51a3-41ac-4bf5-a867-829b21a5be28 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:50:03.781666 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c90d51a3-41ac-4bf5-a867-829b21a5be28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:03.789428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1358/2738] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:50:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:50:03.793917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9d365-74c2-4c4e-b461-3635f93c42c9 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:03.805326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9d365-74c2-4c4e-b461-3635f93c42c9 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:03.809803 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad51bb-cfa6-4de4-a0ac-7c9680df5902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:03.810403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad51bb-cfa6-4de4-a0ac-7c9680df5902 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:03.810680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad51bb-cfa6-4de4-a0ac-7c9680df5902 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:03.810947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ad51bb-cfa6-4de4-a0ac-7c9680df5902 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:03.811590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad51bb-cfa6-4de4-a0ac-7c9680df5902 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:50:03.811889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad51bb-cfa6-4de4-a0ac-7c9680df5902 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:03.813536 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ad51bb-cfa6-4de4-a0ac-7c9680df5902 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:03.826312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1381/2739] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:50:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:50:03.831276 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d51a3-41ac-4bf5-a867-829b21a5be28 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:03.842022 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d51a3-41ac-4bf5-a867-829b21a5be28 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:03.846664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ee4ee-2a6a-4aa2-9576-16f1cc9288a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:03.847257 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ee4ee-2a6a-4aa2-9576-16f1cc9288a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:03.847531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ee4ee-2a6a-4aa2-9576-16f1cc9288a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:03.847796 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0ee4ee-2a6a-4aa2-9576-16f1cc9288a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:03.848414 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ee4ee-2a6a-4aa2-9576-16f1cc9288a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:03.850319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0ee4ee-2a6a-4aa2-9576-16f1cc9288a6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:03.876642 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1359/2740] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:50:03 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:50:03.880748 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad51bb-cfa6-4de4-a0ac-7c9680df5902 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:03.891243 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ad51bb-cfa6-4de4-a0ac-7c9680df5902 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:03.945229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a54ff-a697-4edb-9623-52b3fe5164a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:03.945817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a54ff-a697-4edb-9623-52b3fe5164a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:03.946090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a54ff-a697-4edb-9623-52b3fe5164a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:03.946352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a54ff-a697-4edb-9623-52b3fe5164a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:03.946953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a54ff-a697-4edb-9623-52b3fe5164a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:03.949698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a54ff-a697-4edb-9623-52b3fe5164a5 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:03.970532 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1382/2741] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:50:03 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:50:03.974611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ee4ee-2a6a-4aa2-9576-16f1cc9288a6 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:03.984893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0ee4ee-2a6a-4aa2-9576-16f1cc9288a6 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:03.989602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438409b9-b1c1-40a5-973d-d2edcc4f1307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:03.990200 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438409b9-b1c1-40a5-973d-d2edcc4f1307 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:03.990481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438409b9-b1c1-40a5-973d-d2edcc4f1307 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:03.990755 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438409b9-b1c1-40a5-973d-d2edcc4f1307 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:03.991396 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438409b9-b1c1-40a5-973d-d2edcc4f1307 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:03.993012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438409b9-b1c1-40a5-973d-d2edcc4f1307 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:04.003433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1360/2742] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:50:03 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:50:04.007345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a54ff-a697-4edb-9623-52b3fe5164a5 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:04.018449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a54ff-a697-4edb-9623-52b3fe5164a5 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:04.023040 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4ee2cf-06cd-461c-8736-7db15938369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:04.023625 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ee2cf-06cd-461c-8736-7db15938369f admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:04.023901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ee2cf-06cd-461c-8736-7db15938369f admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:04.024166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4ee2cf-06cd-461c-8736-7db15938369f admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:04.024826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ee2cf-06cd-461c-8736-7db15938369f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:04.026395 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4ee2cf-06cd-461c-8736-7db15938369f admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:04.037523 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1383/2743] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:50:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:50:04.041583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438409b9-b1c1-40a5-973d-d2edcc4f1307 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:04.051676 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438409b9-b1c1-40a5-973d-d2edcc4f1307 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:04.056533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd17ec-3e9a-4dab-b164-63b6571ea8e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:04.057158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd17ec-3e9a-4dab-b164-63b6571ea8e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:04.057438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd17ec-3e9a-4dab-b164-63b6571ea8e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:04.057710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd17ec-3e9a-4dab-b164-63b6571ea8e6 admin admin] PATH_INFO: `/v3/projects/35b7e27fd06c44a7ada160e3ba0685a2/users/2361ab74955440069bcba88debcb2d81/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:04.072873 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd17ec-3e9a-4dab-b164-63b6571ea8e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35b7e27fd06c44a7ada160e3ba0685a2, user_id=2361ab74955440069bcba88debcb2d81, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:04.076327 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd17ec-3e9a-4dab-b164-63b6571ea8e6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:04.088795 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1361/2744] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:50:04 2022] PUT /identity/v3/projects/35b7e27fd06c44a7ada160e3ba0685a2/users/2361ab74955440069bcba88debcb2d81/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:50:04.099928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e9d29-ad87-4c21-aaed-3038b3c3281e None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:04.100213 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e9d29-ad87-4c21-aaed-3038b3c3281e None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:04.100483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658e9d29-ad87-4c21-aaed-3038b3c3281e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:04.129219 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-658e9d29-ad87-4c21-aaed-3038b3c3281e None None] MFA Rules not processed for user `2361ab74955440069bcba88debcb2d81`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:50:04.130417 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-658e9d29-ad87-4c21-aaed-3038b3c3281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:04.196520 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1384/2745] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:50:04 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:50:04.209019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd17ec-3e9a-4dab-b164-63b6571ea8e6 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:04.210825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0ddd17ec-3e9a-4dab-b164-63b6571ea8e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:04.268258 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd17ec-3e9a-4dab-b164-63b6571ea8e6 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:04.272087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a3b2d-2ef0-4672-9d0e-cd4175d643ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42cf265450454ff9a27a29093336205e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:04.272687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a3b2d-2ef0-4672-9d0e-cd4175d643ae service glance] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:04.272993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a3b2d-2ef0-4672-9d0e-cd4175d643ae service glance] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:04.273265 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a3b2d-2ef0-4672-9d0e-cd4175d643ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:04.282193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a3b2d-2ef0-4672-9d0e-cd4175d643ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:04.283607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a3b2d-2ef0-4672-9d0e-cd4175d643ae service glance] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:04.292603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1362/2746] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:50:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:50:14.193622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:50:14.288487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=5,keystone:SELECT=64,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:50:20.174265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e9d29-ad87-4c21-aaed-3038b3c3281e None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:20.175687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-658e9d29-ad87-4c21-aaed-3038b3c3281e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:20.230976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658e9d29-ad87-4c21-aaed-3038b3c3281e None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:20.234492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0623e-dad4-435c-8f80-0b41e5733246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:20.235086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0623e-dad4-435c-8f80-0b41e5733246 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:20.235355 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0623e-dad4-435c-8f80-0b41e5733246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:20.235616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae0623e-dad4-435c-8f80-0b41e5733246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:20.237329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-dae0623e-dad4-435c-8f80-0b41e5733246 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:20.298763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0623e-dad4-435c-8f80-0b41e5733246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:20.300226 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae0623e-dad4-435c-8f80-0b41e5733246 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:20.309707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1385/2747] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:50:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:50:23.427059 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a3b2d-2ef0-4672-9d0e-cd4175d643ae service glance] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:23.428877 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-961a3b2d-2ef0-4672-9d0e-cd4175d643ae service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:23.447017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a3b2d-2ef0-4672-9d0e-cd4175d643ae service glance] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:23.451655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71ab79c-b905-440f-95fb-50d4d76cefad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:23.452280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ab79c-b905-440f-95fb-50d4d76cefad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:23.452555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ab79c-b905-440f-95fb-50d4d76cefad admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:23.452887 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71ab79c-b905-440f-95fb-50d4d76cefad admin admin] PATH_INFO: `/v3/users/e305e7d0cbfc4aeca6164685d0155475` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:23.454050 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ab79c-b905-440f-95fb-50d4d76cefad admin admin] RBAC: Authorizing `identity:delete_user(user_id=e305e7d0cbfc4aeca6164685d0155475)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:23.455987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71ab79c-b905-440f-95fb-50d4d76cefad admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:23.505863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f71ab79c-b905-440f-95fb-50d4d76cefad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e305e7d0cbfc4aeca6164685d0155475'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:23.506506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f71ab79c-b905-440f-95fb-50d4d76cefad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e305e7d0cbfc4aeca6164685d0155475'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:23.514744 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f71ab79c-b905-440f-95fb-50d4d76cefad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e305e7d0cbfc4aeca6164685d0155475'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:23.522550 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f71ab79c-b905-440f-95fb-50d4d76cefad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e305e7d0cbfc4aeca6164685d0155475'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:23.534839 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f71ab79c-b905-440f-95fb-50d4d76cefad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e305e7d0cbfc4aeca6164685d0155475'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:23.540651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1363/2748] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:50:23 2022] DELETE /identity/v3/users/e305e7d0cbfc4aeca6164685d0155475 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:50:23.712567 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0623e-dad4-435c-8f80-0b41e5733246 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:23.723338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae0623e-dad4-435c-8f80-0b41e5733246 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:23.780352 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:23.781007 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:23.781286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:23.781601 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] PATH_INFO: `/v3/projects/38bf7d2d331a472c9d517f3c3067f58e` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:23.782707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38bf7d2d331a472c9d517f3c3067f58e)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:23.785677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:23.830676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38bf7d2d331a472c9d517f3c3067f58e', 'request_id': 'req-6cb05856-a890-4483-a254-b7bc4289b276'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:23.833835 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:50:23.839589 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38bf7d2d331a472c9d517f3c3067f58e', 'request_id': 'req-6cb05856-a890-4483-a254-b7bc4289b276'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:23.846913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38bf7d2d331a472c9d517f3c3067f58e', 'request_id': 'req-6cb05856-a890-4483-a254-b7bc4289b276'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:23.858727 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] The token cache is being invalidate because project 38bf7d2d331a472c9d517f3c3067f58e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:50:23.859015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:23.859636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-6cb05856-a890-4483-a254-b7bc4289b276 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:23.861134 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1386/2749] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:50:23 2022] DELETE /identity/v3/projects/38bf7d2d331a472c9d517f3c3067f58e => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:50:23.866360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fa15e-0275-470e-87aa-a7792bc4eff8 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:23.866641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fa15e-0275-470e-87aa-a7792bc4eff8 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:23.866914 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6fa15e-0275-470e-87aa-a7792bc4eff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:23.886282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-bf6fa15e-0275-470e-87aa-a7792bc4eff8 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:50:23.887472 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-bf6fa15e-0275-470e-87aa-a7792bc4eff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:23.945389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1364/2750] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:50:23.953359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e1ff8-dc34-4851-9fc1-bea985dbba68 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:23.953642 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e1ff8-dc34-4851-9fc1-bea985dbba68 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:23.953914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e1ff8-dc34-4851-9fc1-bea985dbba68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:23.974399 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-f45e1ff8-dc34-4851-9fc1-bea985dbba68 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:50:23.975587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-f45e1ff8-dc34-4851-9fc1-bea985dbba68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:23.983335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1387/2751] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:50:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:50:23.987521 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fa15e-0275-470e-87aa-a7792bc4eff8 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:23.997523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6fa15e-0275-470e-87aa-a7792bc4eff8 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:24.002292 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89311830-ec46-4b1d-a27c-dcc6af2182e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:24.002889 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89311830-ec46-4b1d-a27c-dcc6af2182e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:24.003184 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89311830-ec46-4b1d-a27c-dcc6af2182e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:24.004029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89311830-ec46-4b1d-a27c-dcc6af2182e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:24.004674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89311830-ec46-4b1d-a27c-dcc6af2182e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:50:24.005009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89311830-ec46-4b1d-a27c-dcc6af2182e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:24.006622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89311830-ec46-4b1d-a27c-dcc6af2182e7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:24.018137 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1365/2752] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:50:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:50:24.022820 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e1ff8-dc34-4851-9fc1-bea985dbba68 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:24.032371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e1ff8-dc34-4851-9fc1-bea985dbba68 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:24.037663 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6157092-894f-4fac-baec-a2470d3d84fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:24.038254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6157092-894f-4fac-baec-a2470d3d84fc admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:24.038525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6157092-894f-4fac-baec-a2470d3d84fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:24.038790 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6157092-894f-4fac-baec-a2470d3d84fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:24.039403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6157092-894f-4fac-baec-a2470d3d84fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:24.041306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6157092-894f-4fac-baec-a2470d3d84fc admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:24.072608 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1388/2753] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:50:24 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:50:24.073844 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89311830-ec46-4b1d-a27c-dcc6af2182e7 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:24.085280 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89311830-ec46-4b1d-a27c-dcc6af2182e7 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:24.140706 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66739ff8-edf6-4f36-82aa-bfb7f1082c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:24.141319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66739ff8-edf6-4f36-82aa-bfb7f1082c0f admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:24.141593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66739ff8-edf6-4f36-82aa-bfb7f1082c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:24.141858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66739ff8-edf6-4f36-82aa-bfb7f1082c0f admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:24.142464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66739ff8-edf6-4f36-82aa-bfb7f1082c0f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:24.145170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66739ff8-edf6-4f36-82aa-bfb7f1082c0f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:24.152970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:50:24.167351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1366/2754] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:50:24 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:50:24.171451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6157092-894f-4fac-baec-a2470d3d84fc admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:24.181993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6157092-894f-4fac-baec-a2470d3d84fc admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:24.186492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b5b21-5d82-427b-a397-1468a001adf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:24.187075 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b5b21-5d82-427b-a397-1468a001adf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:24.187349 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b5b21-5d82-427b-a397-1468a001adf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:24.187621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04b5b21-5d82-427b-a397-1468a001adf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:24.188242 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b5b21-5d82-427b-a397-1468a001adf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:24.189837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04b5b21-5d82-427b-a397-1468a001adf5 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:24.200343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1389/2755] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:50:24.204180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66739ff8-edf6-4f36-82aa-bfb7f1082c0f admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:24.214953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66739ff8-edf6-4f36-82aa-bfb7f1082c0f admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:24.219542 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9c097d-28c9-4aef-b890-00755b37a72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:24.220131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c097d-28c9-4aef-b890-00755b37a72f admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:24.220411 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c097d-28c9-4aef-b890-00755b37a72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:24.220680 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9c097d-28c9-4aef-b890-00755b37a72f admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:24.221339 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c097d-28c9-4aef-b890-00755b37a72f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:24.222960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9c097d-28c9-4aef-b890-00755b37a72f admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:24.232674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1367/2756] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:50:24 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:50:24.236832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b5b21-5d82-427b-a397-1468a001adf5 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:24.246885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04b5b21-5d82-427b-a397-1468a001adf5 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:24.251617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7456d53-ef14-4a86-918a-98bf2e16c7ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:24.252269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7456d53-ef14-4a86-918a-98bf2e16c7ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:24.252545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7456d53-ef14-4a86-918a-98bf2e16c7ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:24.252842 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7456d53-ef14-4a86-918a-98bf2e16c7ba admin admin] PATH_INFO: `/v3/projects/121c13a143f447a1822f8df04b1a31a5/users/68515ffe51b74aacb09e8584346e6db3/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:24.268141 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7456d53-ef14-4a86-918a-98bf2e16c7ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=121c13a143f447a1822f8df04b1a31a5, user_id=68515ffe51b74aacb09e8584346e6db3, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:24.271681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7456d53-ef14-4a86-918a-98bf2e16c7ba admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:24.284560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1390/2757] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:50:24 2022] PUT /identity/v3/projects/121c13a143f447a1822f8df04b1a31a5/users/68515ffe51b74aacb09e8584346e6db3/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:50:24.289636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d669-0ad8-4266-bb9f-05e32e3f5f69 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:24.289948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d669-0ad8-4266-bb9f-05e32e3f5f69 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:24.290247 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d42d669-0ad8-4266-bb9f-05e32e3f5f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:24.309035 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-3d42d669-0ad8-4266-bb9f-05e32e3f5f69 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:50:24.310221 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-3d42d669-0ad8-4266-bb9f-05e32e3f5f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:24.368563 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1368/2758] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:50:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:50:24.380399 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7456d53-ef14-4a86-918a-98bf2e16c7ba admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:24.382135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d7456d53-ef14-4a86-918a-98bf2e16c7ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:24.440505 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7456d53-ef14-4a86-918a-98bf2e16c7ba admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:24.449688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb6c03-ab28-409a-825f-9fb2c83f487e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:24.450293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6c03-ab28-409a-825f-9fb2c83f487e service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:24.450620 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6c03-ab28-409a-825f-9fb2c83f487e service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:24.450836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fb6c03-ab28-409a-825f-9fb2c83f487e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:24.460137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb6c03-ab28-409a-825f-9fb2c83f487e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:24.461583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fb6c03-ab28-409a-825f-9fb2c83f487e service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:24.470951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1391/2759] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:50:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:50:28.318205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd764d-c1f7-4d37-9317-5ebf27a1f238 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:28.318720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd764d-c1f7-4d37-9317-5ebf27a1f238 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:28.318959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd764d-c1f7-4d37-9317-5ebf27a1f238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:28.348818 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-29dd764d-c1f7-4d37-9317-5ebf27a1f238 None None] MFA Rules not processed for user `68515ffe51b74aacb09e8584346e6db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:50:28.350131 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-29dd764d-c1f7-4d37-9317-5ebf27a1f238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:28.416326 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1369/2760] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:50:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:50:28.433224 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb6c03-ab28-409a-825f-9fb2c83f487e service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:28.434636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d1fb6c03-ab28-409a-825f-9fb2c83f487e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:28.509655 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fb6c03-ab28-409a-825f-9fb2c83f487e service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:28.515850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0b3d7-3924-4fde-a0e0-80d7660b985e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:28.516458 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0b3d7-3924-4fde-a0e0-80d7660b985e service nova] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:28.516733 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0b3d7-3924-4fde-a0e0-80d7660b985e service nova] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:28.517030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd0b3d7-3924-4fde-a0e0-80d7660b985e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:28.530306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0b3d7-3924-4fde-a0e0-80d7660b985e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:28.532049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd0b3d7-3924-4fde-a0e0-80d7660b985e service nova] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:28.543871 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1392/2761] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:50:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:50:30.149506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fb6ba-f966-4242-be0a-a1fc615fd1eb None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:30.149974 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fb6ba-f966-4242-be0a-a1fc615fd1eb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:30.150081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102fb6ba-f966-4242-be0a-a1fc615fd1eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:30.169897 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-102fb6ba-f966-4242-be0a-a1fc615fd1eb None None] MFA Rules not processed for user `68515ffe51b74aacb09e8584346e6db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:50:30.171089 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-102fb6ba-f966-4242-be0a-a1fc615fd1eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:30.182973 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1370/2762] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:50:30 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:50:36.978496 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0b3d7-3924-4fde-a0e0-80d7660b985e service nova] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:36.980047 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-efd0b3d7-3924-4fde-a0e0-80d7660b985e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:36.990854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:50:36.995673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd0b3d7-3924-4fde-a0e0-80d7660b985e service nova] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:37.001237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:37.002296 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:37.004085 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:37.004451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] PATH_INFO: `/v3/users/2361ab74955440069bcba88debcb2d81` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:37.006005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2361ab74955440069bcba88debcb2d81)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:37.008470 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:37.070900 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2361ab74955440069bcba88debcb2d81'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:37.079872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2361ab74955440069bcba88debcb2d81'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:37.080676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2361ab74955440069bcba88debcb2d81'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:37.095917 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2361ab74955440069bcba88debcb2d81'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:37.104152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2361ab74955440069bcba88debcb2d81'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:37.121018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1393/2763] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:50:36 2022] DELETE /identity/v3/users/2361ab74955440069bcba88debcb2d81 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:50:37.121136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98b95-772e-4e21-8e43-0e38ba3eca75 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:37.121136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98b95-772e-4e21-8e43-0e38ba3eca75 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:37.121136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c98b95-772e-4e21-8e43-0e38ba3eca75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:37.136183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-a1c98b95-772e-4e21-8e43-0e38ba3eca75 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:50:37.137417 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a1c98b95-772e-4e21-8e43-0e38ba3eca75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:37.199540 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1371/2764] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:50:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:50:37.209383 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:37.210795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:37.271862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b579582d-a064-4fea-8f44-c8d931cc6f75 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:37.275755 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3fe4f-29b0-47dc-b9e0-f130a5d2e6cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:37.276360 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3fe4f-29b0-47dc-b9e0-f130a5d2e6cc service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:37.276634 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3fe4f-29b0-47dc-b9e0-f130a5d2e6cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:37.276928 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3fe4f-29b0-47dc-b9e0-f130a5d2e6cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:37.287137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3fe4f-29b0-47dc-b9e0-f130a5d2e6cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:37.288546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de3fe4f-29b0-47dc-b9e0-f130a5d2e6cc service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:37.298583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1394/2765] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:50:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:50:37.570846 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c98b95-772e-4e21-8e43-0e38ba3eca75 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:37.572753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-a1c98b95-772e-4e21-8e43-0e38ba3eca75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:37.583425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:50:37.590891 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c98b95-772e-4e21-8e43-0e38ba3eca75 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:37.598086 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:37.598796 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:37.599075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:37.599343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] PATH_INFO: `/v3/projects/35b7e27fd06c44a7ada160e3ba0685a2` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:37.600487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35b7e27fd06c44a7ada160e3ba0685a2)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:37.602692 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:37.652240 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35b7e27fd06c44a7ada160e3ba0685a2', 'request_id': 'req-186eb422-6e4c-48fa-962d-69211258c883'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:37.658641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35b7e27fd06c44a7ada160e3ba0685a2', 'request_id': 'req-186eb422-6e4c-48fa-962d-69211258c883'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:37.666708 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35b7e27fd06c44a7ada160e3ba0685a2', 'request_id': 'req-186eb422-6e4c-48fa-962d-69211258c883'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:37.678367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] The token cache is being invalidate because project 35b7e27fd06c44a7ada160e3ba0685a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:50:37.678687 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:37.679322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:50:37.681048 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1372/2766] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:50:37 2022] DELETE /identity/v3/projects/35b7e27fd06c44a7ada160e3ba0685a2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:50:47.293972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:50:47.674701 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=26,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:50:50.584879 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3fe4f-29b0-47dc-b9e0-f130a5d2e6cc service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:50.586285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-6de3fe4f-29b0-47dc-b9e0-f130a5d2e6cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:50.626597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3fe4f-29b0-47dc-b9e0-f130a5d2e6cc service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:50.674738 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa66185-79c9-4583-bbde-077a4c7b7ce2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:50.675330 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa66185-79c9-4583-bbde-077a4c7b7ce2 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:50.676078 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa66185-79c9-4583-bbde-077a4c7b7ce2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:50.676356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa66185-79c9-4583-bbde-077a4c7b7ce2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:50.678127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8aa66185-79c9-4583-bbde-077a4c7b7ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:50.748335 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa66185-79c9-4583-bbde-077a4c7b7ce2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:50.749765 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa66185-79c9-4583-bbde-077a4c7b7ce2 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:50.758998 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1395/2767] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:50:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:50:53.246047 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:53.247579 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:53.364119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186eb422-6e4c-48fa-962d-69211258c883 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:53.369275 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93479673-5526-4e2e-be00-10423919a39f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:53.369876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93479673-5526-4e2e-be00-10423919a39f service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:53.370155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93479673-5526-4e2e-be00-10423919a39f service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:53.370427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93479673-5526-4e2e-be00-10423919a39f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:53.372170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-93479673-5526-4e2e-be00-10423919a39f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:53.383945 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93479673-5526-4e2e-be00-10423919a39f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:53.385564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93479673-5526-4e2e-be00-10423919a39f service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:53.395927 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1373/2768] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:50:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:50:55.633988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa66185-79c9-4583-bbde-077a4c7b7ce2 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:50:55.635420 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8aa66185-79c9-4583-bbde-077a4c7b7ce2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:55.647068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa66185-79c9-4583-bbde-077a4c7b7ce2 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:50:55.650616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95574e7-cc19-40ca-9851-cc71da2ec8a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:50:55.651207 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95574e7-cc19-40ca-9851-cc71da2ec8a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:50:55.651506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95574e7-cc19-40ca-9851-cc71da2ec8a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:50:55.651774 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95574e7-cc19-40ca-9851-cc71da2ec8a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:50:55.653475 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d95574e7-cc19-40ca-9851-cc71da2ec8a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:50:55.724387 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=100 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:50:55.738549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95574e7-cc19-40ca-9851-cc71da2ec8a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:50:55.739953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95574e7-cc19-40ca-9851-cc71da2ec8a8 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:50:55.759065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1396/2769] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:50:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:51:01.901329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93479673-5526-4e2e-be00-10423919a39f service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:51:01.902730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-93479673-5526-4e2e-be00-10423919a39f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:51:01.917181 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93479673-5526-4e2e-be00-10423919a39f service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:51:01.921729 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:51:01.922423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:51:01.922761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:51:01.923096 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] PATH_INFO: `/v3/users/bcede7fa39dd47f7ba80c5add2c0f750` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:51:01.924397 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcede7fa39dd47f7ba80c5add2c0f750)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:51:01.927039 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:51:01.978080 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcede7fa39dd47f7ba80c5add2c0f750'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:01.978854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcede7fa39dd47f7ba80c5add2c0f750'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:01.986913 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcede7fa39dd47f7ba80c5add2c0f750'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:01.994738 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcede7fa39dd47f7ba80c5add2c0f750'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.007760 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcede7fa39dd47f7ba80c5add2c0f750'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.013670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1374/2770] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:51:01 2022] DELETE /identity/v3/users/bcede7fa39dd47f7ba80c5add2c0f750 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:51:02.183711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95574e7-cc19-40ca-9851-cc71da2ec8a8 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:51:02.194703 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95574e7-cc19-40ca-9851-cc71da2ec8a8 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:51:02.258114 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:51:02.258734 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:51:02.259009 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:51:02.259274 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] PATH_INFO: `/v3/projects/410f8fc9cc284ba78edf9f4db25f9d31` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:51:02.260374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=410f8fc9cc284ba78edf9f4db25f9d31)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:51:02.262277 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:51:02.308074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '410f8fc9cc284ba78edf9f4db25f9d31', 'request_id': 'req-d08560fd-8f75-498f-b9b7-b58b94c22d1e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.315170 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '410f8fc9cc284ba78edf9f4db25f9d31', 'request_id': 'req-d08560fd-8f75-498f-b9b7-b58b94c22d1e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.320741 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '410f8fc9cc284ba78edf9f4db25f9d31', 'request_id': 'req-d08560fd-8f75-498f-b9b7-b58b94c22d1e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.331398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] The token cache is being invalidate because project 410f8fc9cc284ba78edf9f4db25f9d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:51:02.331735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.332496 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.333985 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1397/2771] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:51:02 2022] DELETE /identity/v3/projects/410f8fc9cc284ba78edf9f4db25f9d31 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:51:02.337937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:51:02.339318 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:51:02.352766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50170f9-77dc-41c9-986c-7f76962ff3b4 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:51:02.365442 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:51:02.411458 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:51:02.412081 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:51:02.412351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:51:02.412621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] PATH_INFO: `/v3/users/365b65fe391445a7a4693d9d445203e9` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:51:02.413766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] RBAC: Authorizing `identity:delete_user(user_id=365b65fe391445a7a4693d9d445203e9)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:51:02.415650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:51:02.460555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365b65fe391445a7a4693d9d445203e9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.461230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365b65fe391445a7a4693d9d445203e9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.468404 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365b65fe391445a7a4693d9d445203e9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.476582 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365b65fe391445a7a4693d9d445203e9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.489103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365b65fe391445a7a4693d9d445203e9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.494837 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1375/2772] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:51:02 2022] DELETE /identity/v3/users/365b65fe391445a7a4693d9d445203e9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:51:02.683258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:51:02.684813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:51:02.698003 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08560fd-8f75-498f-b9b7-b58b94c22d1e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:51:02.754411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:51:02.755031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:51:02.755309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:51:02.755581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] PATH_INFO: `/v3/projects/c6190732d5f743e6ab4e220ea6aad81d` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:51:02.756689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6190732d5f743e6ab4e220ea6aad81d)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:51:02.758591 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:51:02.768611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:51:02.808166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6190732d5f743e6ab4e220ea6aad81d', 'request_id': 'req-e562a5e0-3a86-4561-8802-a30dec72a656'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.815120 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6190732d5f743e6ab4e220ea6aad81d', 'request_id': 'req-e562a5e0-3a86-4561-8802-a30dec72a656'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.820592 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6190732d5f743e6ab4e220ea6aad81d', 'request_id': 'req-e562a5e0-3a86-4561-8802-a30dec72a656'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.831327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] The token cache is being invalidate because project c6190732d5f743e6ab4e220ea6aad81d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:51:02.832306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.833064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e562a5e0-3a86-4561-8802-a30dec72a656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:51:02.834580 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1398/2773] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:51:02 2022] DELETE /identity/v3/projects/c6190732d5f743e6ab4e220ea6aad81d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:51:07.336044 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:51:07.337459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:51:07.392681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756aa5dc-b42b-44f9-80c5-da84460df12c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:51:07.440010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc43d4a5-5a95-46ef-9119-80807b1f10ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:51:07.440531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43d4a5-5a95-46ef-9119-80807b1f10ea service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:51:07.440832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43d4a5-5a95-46ef-9119-80807b1f10ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:51:07.441190 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc43d4a5-5a95-46ef-9119-80807b1f10ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:51:07.443058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-dc43d4a5-5a95-46ef-9119-80807b1f10ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:51:07.448953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:51:07.502118 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43d4a5-5a95-46ef-9119-80807b1f10ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:51:07.504075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc43d4a5-5a95-46ef-9119-80807b1f10ea service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:51:07.513166 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1376/2774] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:51:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:51:12.835768 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:51:14.372612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c797e-68c5-41fd-b928-360ba31c840c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:51:14.373281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c797e-68c5-41fd-b928-360ba31c840c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:51:14.373281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672c797e-68c5-41fd-b928-360ba31c840c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:51:14.391958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-672c797e-68c5-41fd-b928-360ba31c840c None None] MFA Rules not processed for user `29018ec15cc147d3919d9047569a4c68`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:51:14.393189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-672c797e-68c5-41fd-b928-360ba31c840c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:51:14.400278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1399/2775] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:51:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:51:17.509148 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=24 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:51:24.387202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:51:27.338361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb6261-c997-418f-a74f-4772485c3ac9 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:51:27.338906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb6261-c997-418f-a74f-4772485c3ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:51:27.338906 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-badb6261-c997-418f-a74f-4772485c3ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:51:27.359468 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-badb6261-c997-418f-a74f-4772485c3ac9 None None] MFA Rules not processed for user `68515ffe51b74aacb09e8584346e6db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:51:27.360735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-badb6261-c997-418f-a74f-4772485c3ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:51:27.436988 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1377/2776] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:51:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:51:29.748237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c797e-68c5-41fd-b928-360ba31c840c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:51:29.749749 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-672c797e-68c5-41fd-b928-360ba31c840c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:51:29.828307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672c797e-68c5-41fd-b928-360ba31c840c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:51:29.832495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaddecc-0747-4639-af65-c13d40dd0400 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:51:29.833131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaddecc-0747-4639-af65-c13d40dd0400 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:51:29.833410 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaddecc-0747-4639-af65-c13d40dd0400 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:51:29.833681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaddecc-0747-4639-af65-c13d40dd0400 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:51:29.835423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7eaddecc-0747-4639-af65-c13d40dd0400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:51:29.901057 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaddecc-0747-4639-af65-c13d40dd0400 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:51:29.902490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaddecc-0747-4639-af65-c13d40dd0400 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:51:29.913022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1400/2777] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:51:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:51:37.421556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=25 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:51:39.909158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=59 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:52:25.043987 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb670a8-4e46-4b1a-b0ff-a05065cde349 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:52:25.044515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb670a8-4e46-4b1a-b0ff-a05065cde349 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:52:25.044515 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb670a8-4e46-4b1a-b0ff-a05065cde349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:52:25.064289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-afb670a8-4e46-4b1a-b0ff-a05065cde349 None None] MFA Rules not processed for user `68515ffe51b74aacb09e8584346e6db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:52:25.065467 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-afb670a8-4e46-4b1a-b0ff-a05065cde349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:52:25.073355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1378/2778] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:52:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:52:35.059075 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:53:02.707731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaddecc-0747-4639-af65-c13d40dd0400 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:53:02.709885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7eaddecc-0747-4639-af65-c13d40dd0400 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:53:02.721089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaddecc-0747-4639-af65-c13d40dd0400 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:53:02.724724 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d970e-c24f-4c68-b90f-d71434e12dc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:53:02.725364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d970e-c24f-4c68-b90f-d71434e12dc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:53:02.725636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d970e-c24f-4c68-b90f-d71434e12dc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:53:02.725903 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701d970e-c24f-4c68-b90f-d71434e12dc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:53:02.727593 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-701d970e-c24f-4c68-b90f-d71434e12dc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:53:02.821069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d970e-c24f-4c68-b90f-d71434e12dc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:53:02.823099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701d970e-c24f-4c68-b90f-d71434e12dc2 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:53:02.844096 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1401/2779] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:53:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:53:04.027720 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb670a8-4e46-4b1a-b0ff-a05065cde349 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:53:04.037779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb670a8-4e46-4b1a-b0ff-a05065cde349 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:53:04.042330 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ef0b7-9f73-4750-b9dd-90a0ff897319 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:53:04.042922 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef0b7-9f73-4750-b9dd-90a0ff897319 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:53:04.043202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef0b7-9f73-4750-b9dd-90a0ff897319 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:53:04.043480 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ef0b7-9f73-4750-b9dd-90a0ff897319 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:53:04.045351 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-096ef0b7-9f73-4750-b9dd-90a0ff897319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:53:04.059158 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ef0b7-9f73-4750-b9dd-90a0ff897319 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:53:04.060556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ef0b7-9f73-4750-b9dd-90a0ff897319 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:53:04.071229 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1379/2780] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:53:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:53:12.841645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=46 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:53:14.066437 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=6 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:53:15.641196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d970e-c24f-4c68-b90f-d71434e12dc2 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:53:15.649924 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701d970e-c24f-4c68-b90f-d71434e12dc2 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:53:15.654245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7f4c9-feac-4a21-b61d-656bcec3fb82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:53:15.654834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7f4c9-feac-4a21-b61d-656bcec3fb82 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:53:15.655107 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7f4c9-feac-4a21-b61d-656bcec3fb82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:53:15.655371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7f4c9-feac-4a21-b61d-656bcec3fb82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:53:15.657083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-99c7f4c9-feac-4a21-b61d-656bcec3fb82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:53:15.719280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7f4c9-feac-4a21-b61d-656bcec3fb82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:53:15.721558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c7f4c9-feac-4a21-b61d-656bcec3fb82 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:53:15.735498 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1402/2781] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:53:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:53:18.579403 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ef0b7-9f73-4750-b9dd-90a0ff897319 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:53:18.581063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-096ef0b7-9f73-4750-b9dd-90a0ff897319 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:53:18.665807 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ef0b7-9f73-4750-b9dd-90a0ff897319 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:53:18.671141 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3a788-e7ed-47b2-897a-070b86c306b4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:53:18.671746 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3a788-e7ed-47b2-897a-070b86c306b4 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:53:18.672016 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3a788-e7ed-47b2-897a-070b86c306b4 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:53:18.672293 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c3a788-e7ed-47b2-897a-070b86c306b4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:53:18.674065 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-25c3a788-e7ed-47b2-897a-070b86c306b4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:53:18.693727 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3a788-e7ed-47b2-897a-070b86c306b4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:53:18.695329 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c3a788-e7ed-47b2-897a-070b86c306b4 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:53:18.709117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1380/2782] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:53:18 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:53:21.879156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7f4c9-feac-4a21-b61d-656bcec3fb82 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:53:21.880631 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-99c7f4c9-feac-4a21-b61d-656bcec3fb82 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:53:21.893135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c7f4c9-feac-4a21-b61d-656bcec3fb82 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:53:21.897685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:53:21.898338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:53:21.898612 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:53:21.898880 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] PATH_INFO: `/v3/users/29018ec15cc147d3919d9047569a4c68` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:53:21.899990 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] RBAC: Authorizing `identity:delete_user(user_id=29018ec15cc147d3919d9047569a4c68)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:53:21.905560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:53:21.956300 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29018ec15cc147d3919d9047569a4c68'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:21.965266 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29018ec15cc147d3919d9047569a4c68'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:21.965971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29018ec15cc147d3919d9047569a4c68'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:21.977966 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29018ec15cc147d3919d9047569a4c68'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:21.985728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29018ec15cc147d3919d9047569a4c68'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:21.991540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1403/2783] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:53:21 2022] DELETE /identity/v3/users/29018ec15cc147d3919d9047569a4c68 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:53:22.508009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3a788-e7ed-47b2-897a-070b86c306b4 service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:53:22.521641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c3a788-e7ed-47b2-897a-070b86c306b4 service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:53:22.583544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:53:22.584178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:53:22.584456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:53:22.584723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] PATH_INFO: `/v3/projects/cc47231427ec4337833eb5d733b211bc` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:53:22.585895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc47231427ec4337833eb5d733b211bc)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:53:22.587817 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:53:22.632250 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc47231427ec4337833eb5d733b211bc', 'request_id': 'req-85c6999e-3190-4ba5-80b2-8aa68951547e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:22.638866 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc47231427ec4337833eb5d733b211bc', 'request_id': 'req-85c6999e-3190-4ba5-80b2-8aa68951547e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:22.646271 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc47231427ec4337833eb5d733b211bc', 'request_id': 'req-85c6999e-3190-4ba5-80b2-8aa68951547e'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:22.656460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] The token cache is being invalidate because project cc47231427ec4337833eb5d733b211bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:53:22.656743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:22.657789 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:22.659254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1381/2784] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:53:22 2022] DELETE /identity/v3/projects/cc47231427ec4337833eb5d733b211bc => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:53:22.663456 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:53:22.664786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:53:22.677726 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdc5dd-6fca-41b8-a419-51f8f835e37c admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:53:22.733912 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a17359-61fe-4666-9673-99860906cff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:53:22.734528 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a17359-61fe-4666-9673-99860906cff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:53:22.734802 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a17359-61fe-4666-9673-99860906cff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:53:22.735069 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a17359-61fe-4666-9673-99860906cff6 admin admin] PATH_INFO: `/v3/users/8d0a2c6199534477b7aac1154cc53d4c` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:53:22.736189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a17359-61fe-4666-9673-99860906cff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d0a2c6199534477b7aac1154cc53d4c)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:53:22.738113 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a17359-61fe-4666-9673-99860906cff6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:53:22.743269 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:53:22.784787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-13a17359-61fe-4666-9673-99860906cff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d0a2c6199534477b7aac1154cc53d4c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:22.793046 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-13a17359-61fe-4666-9673-99860906cff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d0a2c6199534477b7aac1154cc53d4c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:22.793678 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-13a17359-61fe-4666-9673-99860906cff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d0a2c6199534477b7aac1154cc53d4c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:22.805399 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-13a17359-61fe-4666-9673-99860906cff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d0a2c6199534477b7aac1154cc53d4c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:22.812597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-13a17359-61fe-4666-9673-99860906cff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d0a2c6199534477b7aac1154cc53d4c'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:22.818449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1404/2785] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:53:22 2022] DELETE /identity/v3/users/8d0a2c6199534477b7aac1154cc53d4c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:53:22.991459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:53:22.993032 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:53:23.006469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c6999e-3190-4ba5-80b2-8aa68951547e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:53:23.024845 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:53:23.069202 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:53:23.069829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:53:23.070109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:53:23.070382 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] PATH_INFO: `/v3/projects/91d8ca3743ba41018cc97716004fcce9` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:53:23.071494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91d8ca3743ba41018cc97716004fcce9)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:53:23.074073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:53:23.119038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91d8ca3743ba41018cc97716004fcce9', 'request_id': 'req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:23.126111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91d8ca3743ba41018cc97716004fcce9', 'request_id': 'req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:23.132981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91d8ca3743ba41018cc97716004fcce9', 'request_id': 'req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:23.143993 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] The token cache is being invalidate because project 91d8ca3743ba41018cc97716004fcce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:53:23.144282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:23.144938 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:53:23.146423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1382/2786] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:53:22 2022] DELETE /identity/v3/projects/91d8ca3743ba41018cc97716004fcce9 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:53:27.831609 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71695723-a0c7-45c3-8e31-f2f216bd4490 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:53:27.832123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71695723-a0c7-45c3-8e31-f2f216bd4490 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:53:27.832241 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71695723-a0c7-45c3-8e31-f2f216bd4490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:53:27.851106 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-71695723-a0c7-45c3-8e31-f2f216bd4490 None None] MFA Rules not processed for user `68515ffe51b74aacb09e8584346e6db3`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:53:27.852305 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-71695723-a0c7-45c3-8e31-f2f216bd4490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:53:27.906518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1405/2787] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:53:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:53:33.140962 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:53:37.899573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=48,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:25.369697 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:25.372171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:25.482378 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d60dd-c5c7-4e24-bbe1-9e37ea5296c0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:25.488558 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69499da4-db33-4f00-9178-564bf8fb7ac9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:25.489216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69499da4-db33-4f00-9178-564bf8fb7ac9 service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:25.489494 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69499da4-db33-4f00-9178-564bf8fb7ac9 service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:25.489776 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69499da4-db33-4f00-9178-564bf8fb7ac9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:25.491864 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-69499da4-db33-4f00-9178-564bf8fb7ac9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:25.507217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69499da4-db33-4f00-9178-564bf8fb7ac9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:25.509020 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69499da4-db33-4f00-9178-564bf8fb7ac9 service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:25.523098 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1383/2788] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:54:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:54:34.122833 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71695723-a0c7-45c3-8e31-f2f216bd4490 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:34.124390 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-71695723-a0c7-45c3-8e31-f2f216bd4490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:34.195350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71695723-a0c7-45c3-8e31-f2f216bd4490 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:34.201291 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20baa5bd-d5f2-4bd2-bc34-8e4e1a080a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:34.201945 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa5bd-d5f2-4bd2-bc34-8e4e1a080a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:34.202222 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa5bd-d5f2-4bd2-bc34-8e4e1a080a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:34.202497 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20baa5bd-d5f2-4bd2-bc34-8e4e1a080a08 admin admin] PATH_INFO: `/v3/users/68515ffe51b74aacb09e8584346e6db3` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:34.203671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baa5bd-d5f2-4bd2-bc34-8e4e1a080a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68515ffe51b74aacb09e8584346e6db3)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:34.205784 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20baa5bd-d5f2-4bd2-bc34-8e4e1a080a08 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:34.267325 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-20baa5bd-d5f2-4bd2-bc34-8e4e1a080a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68515ffe51b74aacb09e8584346e6db3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:34.274987 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-20baa5bd-d5f2-4bd2-bc34-8e4e1a080a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68515ffe51b74aacb09e8584346e6db3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:34.275723 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-20baa5bd-d5f2-4bd2-bc34-8e4e1a080a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68515ffe51b74aacb09e8584346e6db3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:34.289268 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-20baa5bd-d5f2-4bd2-bc34-8e4e1a080a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68515ffe51b74aacb09e8584346e6db3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:34.296431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-20baa5bd-d5f2-4bd2-bc34-8e4e1a080a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68515ffe51b74aacb09e8584346e6db3'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:34.302411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1406/2789] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:54:34 2022] DELETE /identity/v3/users/68515ffe51b74aacb09e8584346e6db3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:54:34.479946 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69499da4-db33-4f00-9178-564bf8fb7ac9 service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:34.490857 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69499da4-db33-4f00-9178-564bf8fb7ac9 service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:34.547508 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:34.548136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:34.548409 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:34.548681 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] PATH_INFO: `/v3/projects/121c13a143f447a1822f8df04b1a31a5` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:34.549825 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=121c13a143f447a1822f8df04b1a31a5)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:34.551716 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:34.598355 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '121c13a143f447a1822f8df04b1a31a5', 'request_id': 'req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:34.604338 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '121c13a143f447a1822f8df04b1a31a5', 'request_id': 'req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:34.612101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '121c13a143f447a1822f8df04b1a31a5', 'request_id': 'req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:34.623261 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] The token cache is being invalidate because project 121c13a143f447a1822f8df04b1a31a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:54:34.623548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:34.624174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f2bc8103-e0b7-4f91-b4d9-d53e15454f79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:34.625710 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1384/2790] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:54:34 2022] DELETE /identity/v3/projects/121c13a143f447a1822f8df04b1a31a5 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:34.630674 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297268-a894-4637-b1ce-ee2250cf2ac2 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:34.630962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297268-a894-4637-b1ce-ee2250cf2ac2 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:34.631235 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297268-a894-4637-b1ce-ee2250cf2ac2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:34.690426 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c4297268-a894-4637-b1ce-ee2250cf2ac2 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:34.691549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c4297268-a894-4637-b1ce-ee2250cf2ac2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:34.734853 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:34.753312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1407/2791] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:34.761052 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c307f-9e9d-40d9-b4ca-9ee77151b991 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:34.761336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c307f-9e9d-40d9-b4ca-9ee77151b991 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:34.761607 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811c307f-9e9d-40d9-b4ca-9ee77151b991 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:34.782332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-811c307f-9e9d-40d9-b4ca-9ee77151b991 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:34.783524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-811c307f-9e9d-40d9-b4ca-9ee77151b991 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:34.791211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1385/2792] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:34.795439 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4297268-a894-4637-b1ce-ee2250cf2ac2 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:34.805180 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4297268-a894-4637-b1ce-ee2250cf2ac2 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:34.810130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81268-dfa1-453b-ad1f-364b49fb7f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:34.810728 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81268-dfa1-453b-ad1f-364b49fb7f95 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:34.811005 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81268-dfa1-453b-ad1f-364b49fb7f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:34.811278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd81268-dfa1-453b-ad1f-364b49fb7f95 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:34.811972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd81268-dfa1-453b-ad1f-364b49fb7f95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:54:34.812278 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd81268-dfa1-453b-ad1f-364b49fb7f95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:34.813923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd81268-dfa1-453b-ad1f-364b49fb7f95 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:34.825951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1408/2793] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:54:34.830255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c307f-9e9d-40d9-b4ca-9ee77151b991 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:34.839254 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811c307f-9e9d-40d9-b4ca-9ee77151b991 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:34.844848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b7f1b-c7f1-4b8c-8b89-9ab23fe5e45b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:34.845459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b7f1b-c7f1-4b8c-8b89-9ab23fe5e45b admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:34.845731 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b7f1b-c7f1-4b8c-8b89-9ab23fe5e45b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:34.846022 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077b7f1b-c7f1-4b8c-8b89-9ab23fe5e45b admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:34.846643 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b7f1b-c7f1-4b8c-8b89-9ab23fe5e45b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:34.848503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077b7f1b-c7f1-4b8c-8b89-9ab23fe5e45b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:34.853523 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:34.888645 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1386/2794] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:54:34 2022] POST /identity/v3/projects => generated 379 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:54:34.892715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81268-dfa1-453b-ad1f-364b49fb7f95 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:34.903636 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd81268-dfa1-453b-ad1f-364b49fb7f95 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:34.958846 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f19ba-9657-4728-ac71-701e0df3e269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:34.959362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f19ba-9657-4728-ac71-701e0df3e269 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:34.959635 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f19ba-9657-4728-ac71-701e0df3e269 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:34.959901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f19ba-9657-4728-ac71-701e0df3e269 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:34.960522 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f19ba-9657-4728-ac71-701e0df3e269 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:34.962423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f19ba-9657-4728-ac71-701e0df3e269 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:34.983403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1409/2795] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:54:34 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:54:34.987667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b7f1b-c7f1-4b8c-8b89-9ab23fe5e45b admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:34.998788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077b7f1b-c7f1-4b8c-8b89-9ab23fe5e45b admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:35.003464 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871426a-f628-4f52-9c23-88cb21e2e953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:35.004053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871426a-f628-4f52-9c23-88cb21e2e953 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.004331 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871426a-f628-4f52-9c23-88cb21e2e953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.004597 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b871426a-f628-4f52-9c23-88cb21e2e953 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.005261 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871426a-f628-4f52-9c23-88cb21e2e953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:35.006852 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b871426a-f628-4f52-9c23-88cb21e2e953 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:35.017980 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1387/2796] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:54:35.021506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f19ba-9657-4728-ac71-701e0df3e269 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:35.032890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f19ba-9657-4728-ac71-701e0df3e269 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:35.037446 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325ae13e-fde0-4139-a142-b0b7b3eca1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:35.038032 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ae13e-fde0-4139-a142-b0b7b3eca1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.038309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ae13e-fde0-4139-a142-b0b7b3eca1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.038579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325ae13e-fde0-4139-a142-b0b7b3eca1f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.039211 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ae13e-fde0-4139-a142-b0b7b3eca1f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:35.040811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325ae13e-fde0-4139-a142-b0b7b3eca1f6 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:35.051682 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1410/2797] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:54:35.055764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871426a-f628-4f52-9c23-88cb21e2e953 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:35.066282 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b871426a-f628-4f52-9c23-88cb21e2e953 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:35.070904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea60983-8a3d-4923-9fbc-8646b6a657ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:35.071547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea60983-8a3d-4923-9fbc-8646b6a657ae admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.071827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea60983-8a3d-4923-9fbc-8646b6a657ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.072099 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea60983-8a3d-4923-9fbc-8646b6a657ae admin admin] PATH_INFO: `/v3/projects/4a4e9f466fc1429aa2ab28c0ffe8419d/users/ccf5a08224ef4e908b875c15c36993eb/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.087177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea60983-8a3d-4923-9fbc-8646b6a657ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a4e9f466fc1429aa2ab28c0ffe8419d, user_id=ccf5a08224ef4e908b875c15c36993eb, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:35.089903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea60983-8a3d-4923-9fbc-8646b6a657ae admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:35.103115 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1388/2798] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:54:35 2022] PUT /identity/v3/projects/4a4e9f466fc1429aa2ab28c0ffe8419d/users/ccf5a08224ef4e908b875c15c36993eb/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:35.114649 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47957cbd-dbdf-477a-a9bb-1f1b807a2e81 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.114931 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47957cbd-dbdf-477a-a9bb-1f1b807a2e81 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.115204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47957cbd-dbdf-477a-a9bb-1f1b807a2e81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.143512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-47957cbd-dbdf-477a-a9bb-1f1b807a2e81 None None] MFA Rules not processed for user `ccf5a08224ef4e908b875c15c36993eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:35.144708 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-47957cbd-dbdf-477a-a9bb-1f1b807a2e81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:35.207328 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1411/2799] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:35.211900 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea60983-8a3d-4923-9fbc-8646b6a657ae admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:35.272588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea60983-8a3d-4923-9fbc-8646b6a657ae admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:35.277764 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6b7e5-b1f8-429c-a49d-1bf38de80421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:35.277978 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6b7e5-b1f8-429c-a49d-1bf38de80421 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.278232 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6b7e5-b1f8-429c-a49d-1bf38de80421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.278441 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf6b7e5-b1f8-429c-a49d-1bf38de80421 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.279074 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6b7e5-b1f8-429c-a49d-1bf38de80421 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:35.280996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf6b7e5-b1f8-429c-a49d-1bf38de80421 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:35.311904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1389/2800] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:54:35 2022] POST /identity/v3/projects => generated 381 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:54:35.313555 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47957cbd-dbdf-477a-a9bb-1f1b807a2e81 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:35.325152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47957cbd-dbdf-477a-a9bb-1f1b807a2e81 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:35.368921 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:35.391115 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568bb5c-dc91-4678-9fd1-9157f624d5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:35.391739 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568bb5c-dc91-4678-9fd1-9157f624d5fe admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.392016 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568bb5c-dc91-4678-9fd1-9157f624d5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.392315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568bb5c-dc91-4678-9fd1-9157f624d5fe admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.392972 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568bb5c-dc91-4678-9fd1-9157f624d5fe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:35.395155 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568bb5c-dc91-4678-9fd1-9157f624d5fe admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:35.415673 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1412/2801] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:54:35 2022] POST /identity/v3/users => generated 297 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:54:35.420722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6b7e5-b1f8-429c-a49d-1bf38de80421 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:35.430909 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf6b7e5-b1f8-429c-a49d-1bf38de80421 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:35.436659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c19279-ff21-4db5-9998-8758ecd80e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:35.437279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c19279-ff21-4db5-9998-8758ecd80e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.437556 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c19279-ff21-4db5-9998-8758ecd80e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.437826 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c19279-ff21-4db5-9998-8758ecd80e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.438454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c19279-ff21-4db5-9998-8758ecd80e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:35.440037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c19279-ff21-4db5-9998-8758ecd80e2d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:35.451012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1390/2802] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:54:35.455054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568bb5c-dc91-4678-9fd1-9157f624d5fe admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:35.464814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568bb5c-dc91-4678-9fd1-9157f624d5fe admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:35.469296 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de500411-fe45-4678-827a-082693370773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:35.469929 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de500411-fe45-4678-827a-082693370773 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.470204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de500411-fe45-4678-827a-082693370773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.470472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de500411-fe45-4678-827a-082693370773 admin admin] PATH_INFO: `/v3/projects/b8a1811602e84ce893e607044869c4bd/users/4d21f859b60f4bb981050b8972b7bb18/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.485362 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de500411-fe45-4678-827a-082693370773 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8a1811602e84ce893e607044869c4bd, user_id=4d21f859b60f4bb981050b8972b7bb18, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:35.487960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de500411-fe45-4678-827a-082693370773 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:35.501455 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1413/2803] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:54:35 2022] PUT /identity/v3/projects/b8a1811602e84ce893e607044869c4bd/users/4d21f859b60f4bb981050b8972b7bb18/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:35.505360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c19279-ff21-4db5-9998-8758ecd80e2d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:35.515640 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c19279-ff21-4db5-9998-8758ecd80e2d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:35.572833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f81d5a-76f6-4a90-a691-70d2478d354d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:35.573450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81d5a-76f6-4a90-a691-70d2478d354d admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.573725 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81d5a-76f6-4a90-a691-70d2478d354d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.573996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f81d5a-76f6-4a90-a691-70d2478d354d admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.574761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f81d5a-76f6-4a90-a691-70d2478d354d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:35.576362 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f81d5a-76f6-4a90-a691-70d2478d354d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:35.586334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1391/2804] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:35 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:54:35.590670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de500411-fe45-4678-827a-082693370773 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:35.600789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de500411-fe45-4678-827a-082693370773 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:35.605933 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dafa1-c3c0-49f5-9039-a179fa4b4b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:35.606572 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dafa1-c3c0-49f5-9039-a179fa4b4b50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.606847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dafa1-c3c0-49f5-9039-a179fa4b4b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.607117 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7dafa1-c3c0-49f5-9039-a179fa4b4b50 admin admin] PATH_INFO: `/v3/projects/b8a1811602e84ce893e607044869c4bd/users/4d21f859b60f4bb981050b8972b7bb18/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.609070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dafa1-c3c0-49f5-9039-a179fa4b4b50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8a1811602e84ce893e607044869c4bd, user_id=4d21f859b60f4bb981050b8972b7bb18, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:35.611670 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7dafa1-c3c0-49f5-9039-a179fa4b4b50 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:35.622190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1414/2805] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:54:35 2022] PUT /identity/v3/projects/b8a1811602e84ce893e607044869c4bd/users/4d21f859b60f4bb981050b8972b7bb18/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:35.632262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7341b-9267-4e57-af12-6e9aa452943a None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.632547 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7341b-9267-4e57-af12-6e9aa452943a None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.632850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7341b-9267-4e57-af12-6e9aa452943a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.658153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:35.662132 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-aec7341b-9267-4e57-af12-6e9aa452943a None None] MFA Rules not processed for user `4d21f859b60f4bb981050b8972b7bb18`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:35.663981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-aec7341b-9267-4e57-af12-6e9aa452943a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:35.732712 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1392/2806] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:35 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:35.743979 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dafa1-c3c0-49f5-9039-a179fa4b4b50 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:35.745809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9e7dafa1-c3c0-49f5-9039-a179fa4b4b50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:35.824927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7dafa1-c3c0-49f5-9039-a179fa4b4b50 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:35.828579 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f68a0c-a49d-479a-8efd-a143f7e36483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:35.829202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f68a0c-a49d-479a-8efd-a143f7e36483 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.829476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f68a0c-a49d-479a-8efd-a143f7e36483 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.829744 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f68a0c-a49d-479a-8efd-a143f7e36483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.838511 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f68a0c-a49d-479a-8efd-a143f7e36483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:35.840560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f68a0c-a49d-479a-8efd-a143f7e36483 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:35.849922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1415/2807] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:54:35 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:54:35.925863 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7341b-9267-4e57-af12-6e9aa452943a None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:35.935842 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7341b-9267-4e57-af12-6e9aa452943a None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:35.939435 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eebd75-8985-4476-b1b0-19df8a5e371c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:35.940034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eebd75-8985-4476-b1b0-19df8a5e371c service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:35.940312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eebd75-8985-4476-b1b0-19df8a5e371c service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:35.940580 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eebd75-8985-4476-b1b0-19df8a5e371c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:35.997193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eebd75-8985-4476-b1b0-19df8a5e371c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:35.998541 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eebd75-8985-4476-b1b0-19df8a5e371c service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:36.007347 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1393/2808] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:54:35 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:54:44.836814 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f68a0c-a49d-479a-8efd-a143f7e36483 service cinder] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:44.898689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f68a0c-a49d-479a-8efd-a143f7e36483 service cinder] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:44.903313 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:44.903932 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:44.904204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:44.904472 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] PATH_INFO: `/v3/users/ccf5a08224ef4e908b875c15c36993eb` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:44.905610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccf5a08224ef4e908b875c15c36993eb)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:44.907736 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:44.928495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:44.954136 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccf5a08224ef4e908b875c15c36993eb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:44.961701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccf5a08224ef4e908b875c15c36993eb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:44.962260 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccf5a08224ef4e908b875c15c36993eb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:44.973889 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccf5a08224ef4e908b875c15c36993eb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:44.980958 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccf5a08224ef4e908b875c15c36993eb'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:44.986861 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1416/2809] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:54:44 2022] DELETE /identity/v3/users/ccf5a08224ef4e908b875c15c36993eb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:44.991648 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6862b-020e-4ef8-9721-a0b6cd4c8d26 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:44.992117 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6862b-020e-4ef8-9721-a0b6cd4c8d26 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:44.992239 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb6862b-020e-4ef8-9721-a0b6cd4c8d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:45.012912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-0bb6862b-020e-4ef8-9721-a0b6cd4c8d26 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:45.014108 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0bb6862b-020e-4ef8-9721-a0b6cd4c8d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:45.071400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1394/2810] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:45.081737 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:45.083128 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:45.140597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8421befe-a56e-4a2d-bf6e-89d781d541dc admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:45.144216 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca74f3-6b45-4f01-8822-ccc2312f3852 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:45.144836 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca74f3-6b45-4f01-8822-ccc2312f3852 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:45.145112 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca74f3-6b45-4f01-8822-ccc2312f3852 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:45.145376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ca74f3-6b45-4f01-8822-ccc2312f3852 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:45.155004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca74f3-6b45-4f01-8822-ccc2312f3852 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:45.156702 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ca74f3-6b45-4f01-8822-ccc2312f3852 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:45.165924 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1417/2811] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:54:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:54:45.429771 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6862b-020e-4ef8-9721-a0b6cd4c8d26 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:45.431194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0bb6862b-020e-4ef8-9721-a0b6cd4c8d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:45.444207 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb6862b-020e-4ef8-9721-a0b6cd4c8d26 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:45.449700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:45.450575 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:45.450955 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:45.451291 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] PATH_INFO: `/v3/projects/4a4e9f466fc1429aa2ab28c0ffe8419d` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:45.452659 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a4e9f466fc1429aa2ab28c0ffe8419d)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:45.454995 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:45.491174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:45.504080 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a4e9f466fc1429aa2ab28c0ffe8419d', 'request_id': 'req-f39ef80b-5b4d-4de9-a431-8be10bad944d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:45.510281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a4e9f466fc1429aa2ab28c0ffe8419d', 'request_id': 'req-f39ef80b-5b4d-4de9-a431-8be10bad944d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:45.517835 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a4e9f466fc1429aa2ab28c0ffe8419d', 'request_id': 'req-f39ef80b-5b4d-4de9-a431-8be10bad944d'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:45.529336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] The token cache is being invalidate because project 4a4e9f466fc1429aa2ab28c0ffe8419d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:54:45.529617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:45.530238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:45.531964 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1395/2812] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:54:45 2022] DELETE /identity/v3/projects/4a4e9f466fc1429aa2ab28c0ffe8419d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:45.535994 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca74f3-6b45-4f01-8822-ccc2312f3852 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:45.545777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ca74f3-6b45-4f01-8822-ccc2312f3852 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:45.602303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdd9fabe-083d-4950-be09-600c56d01ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:45.602920 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9fabe-083d-4950-be09-600c56d01ab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:45.603195 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9fabe-083d-4950-be09-600c56d01ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:45.603462 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdd9fabe-083d-4950-be09-600c56d01ab2 admin admin] PATH_INFO: `/v3/users/4d21f859b60f4bb981050b8972b7bb18` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:45.604569 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9fabe-083d-4950-be09-600c56d01ab2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d21f859b60f4bb981050b8972b7bb18)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:45.606590 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdd9fabe-083d-4950-be09-600c56d01ab2 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:45.650272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:45.653204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bdd9fabe-083d-4950-be09-600c56d01ab2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d21f859b60f4bb981050b8972b7bb18'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:45.661248 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bdd9fabe-083d-4950-be09-600c56d01ab2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d21f859b60f4bb981050b8972b7bb18'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:45.662214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bdd9fabe-083d-4950-be09-600c56d01ab2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d21f859b60f4bb981050b8972b7bb18'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:45.673923 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bdd9fabe-083d-4950-be09-600c56d01ab2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d21f859b60f4bb981050b8972b7bb18'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:45.680489 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-bdd9fabe-083d-4950-be09-600c56d01ab2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d21f859b60f4bb981050b8972b7bb18'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:45.686641 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1418/2813] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:54:45 2022] DELETE /identity/v3/users/4d21f859b60f4bb981050b8972b7bb18 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:45.962860 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:45.965009 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:45.978123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ef80b-5b4d-4de9-a431-8be10bad944d admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:46.036383 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:46.037127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.037475 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.037672 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] PATH_INFO: `/v3/projects/b8a1811602e84ce893e607044869c4bd` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.038783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8a1811602e84ce893e607044869c4bd)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:46.040665 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:46.086123 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8a1811602e84ce893e607044869c4bd', 'request_id': 'req-751c50bc-7c10-4f98-bc91-76559a166e0b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:46.092673 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8a1811602e84ce893e607044869c4bd', 'request_id': 'req-751c50bc-7c10-4f98-bc91-76559a166e0b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:46.100015 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8a1811602e84ce893e607044869c4bd', 'request_id': 'req-751c50bc-7c10-4f98-bc91-76559a166e0b'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:46.110743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] The token cache is being invalidate because project b8a1811602e84ce893e607044869c4bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:54:46.111029 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:46.111679 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-751c50bc-7c10-4f98-bc91-76559a166e0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:46.113248 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1396/2814] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:54:45 2022] DELETE /identity/v3/projects/b8a1811602e84ce893e607044869c4bd => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:46.118041 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811585f-b785-4cac-aa4a-01febc4319b9 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.118521 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811585f-b785-4cac-aa4a-01febc4319b9 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.118623 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1811585f-b785-4cac-aa4a-01febc4319b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.138172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-1811585f-b785-4cac-aa4a-01febc4319b9 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:46.139346 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-1811585f-b785-4cac-aa4a-01febc4319b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:46.197975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1419/2815] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 06 22:54:46.206691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0ecb-0b3d-4f7a-a93e-13e6e757e654 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.206970 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0ecb-0b3d-4f7a-a93e-13e6e757e654 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.207244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f0ecb-0b3d-4f7a-a93e-13e6e757e654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.227588 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-768f0ecb-0b3d-4f7a-a93e-13e6e757e654 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:46.228810 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-768f0ecb-0b3d-4f7a-a93e-13e6e757e654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:46.237260 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1397/2816] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:46.241293 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811585f-b785-4cac-aa4a-01febc4319b9 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:46.251061 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1811585f-b785-4cac-aa4a-01febc4319b9 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:46.255687 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a136ef-fbf4-46c8-a006-45b25b781959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:46.256282 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a136ef-fbf4-46c8-a006-45b25b781959 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.256560 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a136ef-fbf4-46c8-a006-45b25b781959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.256854 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a136ef-fbf4-46c8-a006-45b25b781959 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.257495 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a136ef-fbf4-46c8-a006-45b25b781959 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:54:46.257796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a136ef-fbf4-46c8-a006-45b25b781959 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:46.259404 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a136ef-fbf4-46c8-a006-45b25b781959 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:46.271827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1420/2817] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:54:46.275894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f0ecb-0b3d-4f7a-a93e-13e6e757e654 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:46.286375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f0ecb-0b3d-4f7a-a93e-13e6e757e654 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:46.291119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c249df-ac12-45e2-8058-05972166edce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:46.291650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c249df-ac12-45e2-8058-05972166edce admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.291926 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c249df-ac12-45e2-8058-05972166edce admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.292193 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c249df-ac12-45e2-8058-05972166edce admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.293412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c249df-ac12-45e2-8058-05972166edce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:46.295297 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c249df-ac12-45e2-8058-05972166edce admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:46.322538 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1398/2818] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:54:46 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:54:46.327099 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a136ef-fbf4-46c8-a006-45b25b781959 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:46.337131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a136ef-fbf4-46c8-a006-45b25b781959 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:46.392246 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5d2b0-35f6-47e7-92c1-5ca71ee7459a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:46.392860 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5d2b0-35f6-47e7-92c1-5ca71ee7459a admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.393133 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5d2b0-35f6-47e7-92c1-5ca71ee7459a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.393398 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5d2b0-35f6-47e7-92c1-5ca71ee7459a admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.394022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5d2b0-35f6-47e7-92c1-5ca71ee7459a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:46.395893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb5d2b0-35f6-47e7-92c1-5ca71ee7459a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:46.416898 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1421/2819] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:54:46 2022] POST /identity/v3/users => generated 285 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:54:46.422346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c249df-ac12-45e2-8058-05972166edce admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:46.432562 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c249df-ac12-45e2-8058-05972166edce admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:46.437220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4dcd15-5540-44f3-898b-16c8c3b856a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:46.437808 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dcd15-5540-44f3-898b-16c8c3b856a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.438080 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dcd15-5540-44f3-898b-16c8c3b856a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.438343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4dcd15-5540-44f3-898b-16c8c3b856a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.438965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4dcd15-5540-44f3-898b-16c8c3b856a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:46.440537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4dcd15-5540-44f3-898b-16c8c3b856a5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:46.450534 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1399/2820] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:54:46.454411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5d2b0-35f6-47e7-92c1-5ca71ee7459a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:46.464004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5d2b0-35f6-47e7-92c1-5ca71ee7459a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:46.469238 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15afdaf5-8cfd-4ba9-8942-c910b34f0f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:46.469817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afdaf5-8cfd-4ba9-8942-c910b34f0f27 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.470091 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afdaf5-8cfd-4ba9-8942-c910b34f0f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.470358 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15afdaf5-8cfd-4ba9-8942-c910b34f0f27 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.471097 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afdaf5-8cfd-4ba9-8942-c910b34f0f27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:46.472656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15afdaf5-8cfd-4ba9-8942-c910b34f0f27 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:46.484688 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1422/2821] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:54:46.489061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4dcd15-5540-44f3-898b-16c8c3b856a5 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:46.499366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4dcd15-5540-44f3-898b-16c8c3b856a5 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:46.504127 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b2d61-f216-4257-8e17-e81a9dfdbe36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:46.504840 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b2d61-f216-4257-8e17-e81a9dfdbe36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.505120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b2d61-f216-4257-8e17-e81a9dfdbe36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.505395 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639b2d61-f216-4257-8e17-e81a9dfdbe36 admin admin] PATH_INFO: `/v3/projects/c34610acfe9b4072adfae6bbdcdf88c8/users/7154cc7a6c0f45e9991530fe8f3465f8/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.513983 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:46.521459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b2d61-f216-4257-8e17-e81a9dfdbe36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c34610acfe9b4072adfae6bbdcdf88c8, user_id=7154cc7a6c0f45e9991530fe8f3465f8, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:46.524598 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639b2d61-f216-4257-8e17-e81a9dfdbe36 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:46.538278 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1400/2822] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:54:46 2022] PUT /identity/v3/projects/c34610acfe9b4072adfae6bbdcdf88c8/users/7154cc7a6c0f45e9991530fe8f3465f8/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:54:46.550598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb5758-3c18-47df-9911-bd328867d332 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.550809 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb5758-3c18-47df-9911-bd328867d332 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.551079 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86bb5758-3c18-47df-9911-bd328867d332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.578826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-86bb5758-3c18-47df-9911-bd328867d332 None None] MFA Rules not processed for user `7154cc7a6c0f45e9991530fe8f3465f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:46.580035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-86bb5758-3c18-47df-9911-bd328867d332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:46.593264 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:46.645973 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1423/2823] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:46.650573 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b2d61-f216-4257-8e17-e81a9dfdbe36 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:46.711639 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639b2d61-f216-4257-8e17-e81a9dfdbe36 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:46.716456 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0651c24-b2fa-455d-be0f-ce83ccaf6962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:46.717073 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0651c24-b2fa-455d-be0f-ce83ccaf6962 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.717349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0651c24-b2fa-455d-be0f-ce83ccaf6962 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.717617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0651c24-b2fa-455d-be0f-ce83ccaf6962 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.718256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0651c24-b2fa-455d-be0f-ce83ccaf6962 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:46.720134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0651c24-b2fa-455d-be0f-ce83ccaf6962 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:46.746912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1401/2824] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:54:46 2022] POST /identity/v3/projects => generated 369 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:54:46.750983 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb5758-3c18-47df-9911-bd328867d332 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:46.762295 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86bb5758-3c18-47df-9911-bd328867d332 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:46.821823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1d4ed-3173-4500-b63b-250e20e27c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:46.822346 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1d4ed-3173-4500-b63b-250e20e27c06 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.822622 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1d4ed-3173-4500-b63b-250e20e27c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.822890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d1d4ed-3173-4500-b63b-250e20e27c06 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.823499 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1d4ed-3173-4500-b63b-250e20e27c06 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:46.825985 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d1d4ed-3173-4500-b63b-250e20e27c06 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:46.847875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1424/2825] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:54:46 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:54:46.851872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0651c24-b2fa-455d-be0f-ce83ccaf6962 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:46.862201 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0651c24-b2fa-455d-be0f-ce83ccaf6962 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:46.866800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a041e-4f24-4377-921e-cef1f91d5151 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:46.867385 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a041e-4f24-4377-921e-cef1f91d5151 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.867658 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a041e-4f24-4377-921e-cef1f91d5151 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.867925 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8a041e-4f24-4377-921e-cef1f91d5151 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.868554 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a041e-4f24-4377-921e-cef1f91d5151 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:46.870162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8a041e-4f24-4377-921e-cef1f91d5151 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:46.880525 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1402/2826] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:54:46.884435 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1d4ed-3173-4500-b63b-250e20e27c06 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:46.895299 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d1d4ed-3173-4500-b63b-250e20e27c06 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:46.900035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ca49f-6562-4638-9fc8-6d6b8e312143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:46.900677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ca49f-6562-4638-9fc8-6d6b8e312143 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:46.900988 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ca49f-6562-4638-9fc8-6d6b8e312143 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:46.901376 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874ca49f-6562-4638-9fc8-6d6b8e312143 admin admin] PATH_INFO: `/v3/projects/07953e276bf842bbb8ffa662fe1d3104/users/00357c9f83784103ae8b138c23627eef/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:46.916653 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ca49f-6562-4638-9fc8-6d6b8e312143 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07953e276bf842bbb8ffa662fe1d3104, user_id=00357c9f83784103ae8b138c23627eef, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:46.919574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874ca49f-6562-4638-9fc8-6d6b8e312143 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:46.932353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1425/2827] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:54:46 2022] PUT /identity/v3/projects/07953e276bf842bbb8ffa662fe1d3104/users/00357c9f83784103ae8b138c23627eef/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:46.936238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a041e-4f24-4377-921e-cef1f91d5151 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:46.946312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8a041e-4f24-4377-921e-cef1f91d5151 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:47.002779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340f3496-0949-4d75-8619-0aadb06ace89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:47.003792 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f3496-0949-4d75-8619-0aadb06ace89 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:47.004295 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f3496-0949-4d75-8619-0aadb06ace89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:47.004592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340f3496-0949-4d75-8619-0aadb06ace89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:47.005264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f3496-0949-4d75-8619-0aadb06ace89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:47.006850 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-340f3496-0949-4d75-8619-0aadb06ace89 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:47.016723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1403/2828] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:46 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:54:47.020890 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ca49f-6562-4638-9fc8-6d6b8e312143 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:47.031914 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874ca49f-6562-4638-9fc8-6d6b8e312143 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:47.036476 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700724a-1f60-4307-8d1c-9cad5747d093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:47.037152 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700724a-1f60-4307-8d1c-9cad5747d093 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:47.037430 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700724a-1f60-4307-8d1c-9cad5747d093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:47.037697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d700724a-1f60-4307-8d1c-9cad5747d093 admin admin] PATH_INFO: `/v3/projects/07953e276bf842bbb8ffa662fe1d3104/users/00357c9f83784103ae8b138c23627eef/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:47.039587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700724a-1f60-4307-8d1c-9cad5747d093 admin admin] RBAC: Authorizing `identity:create_grant(project_id=07953e276bf842bbb8ffa662fe1d3104, user_id=00357c9f83784103ae8b138c23627eef, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:47.042209 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d700724a-1f60-4307-8d1c-9cad5747d093 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:47.052771 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1426/2829] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:54:47 2022] PUT /identity/v3/projects/07953e276bf842bbb8ffa662fe1d3104/users/00357c9f83784103ae8b138c23627eef/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:47.062869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31694dd0-d442-4f34-8ca1-2bdc30876f12 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:47.063386 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31694dd0-d442-4f34-8ca1-2bdc30876f12 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:47.067017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31694dd0-d442-4f34-8ca1-2bdc30876f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:47.094829 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-31694dd0-d442-4f34-8ca1-2bdc30876f12 None None] MFA Rules not processed for user `00357c9f83784103ae8b138c23627eef`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:47.096025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-31694dd0-d442-4f34-8ca1-2bdc30876f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:47.139991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:47.163555 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1404/2830] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:47.174022 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700724a-1f60-4307-8d1c-9cad5747d093 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:47.175759 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d700724a-1f60-4307-8d1c-9cad5747d093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:47.232270 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d700724a-1f60-4307-8d1c-9cad5747d093 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:47.236173 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc6dd3d-c25a-417a-816f-1cc1504c3525 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:47.236783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6dd3d-c25a-417a-816f-1cc1504c3525 service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:47.237074 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6dd3d-c25a-417a-816f-1cc1504c3525 service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:47.237343 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc6dd3d-c25a-417a-816f-1cc1504c3525 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:47.246838 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6dd3d-c25a-417a-816f-1cc1504c3525 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:47.248245 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc6dd3d-c25a-417a-816f-1cc1504c3525 service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:47.257597 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1427/2831] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:54:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:54:48.603134 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31694dd0-d442-4f34-8ca1-2bdc30876f12 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:48.664854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31694dd0-d442-4f34-8ca1-2bdc30876f12 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:48.669377 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:48.669996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:48.670270 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:48.670535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] PATH_INFO: `/v3/users/7154cc7a6c0f45e9991530fe8f3465f8` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:48.671683 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7154cc7a6c0f45e9991530fe8f3465f8)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:48.673605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:48.720136 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7154cc7a6c0f45e9991530fe8f3465f8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:48.720800 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7154cc7a6c0f45e9991530fe8f3465f8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:48.727690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7154cc7a6c0f45e9991530fe8f3465f8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:48.735219 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7154cc7a6c0f45e9991530fe8f3465f8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:48.747819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7154cc7a6c0f45e9991530fe8f3465f8'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:48.753670 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1405/2832] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:54:48 2022] DELETE /identity/v3/users/7154cc7a6c0f45e9991530fe8f3465f8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:48.758312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd4e59-f109-43af-8a6f-ac865b3f8843 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:48.759189 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd4e59-f109-43af-8a6f-ac865b3f8843 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:48.759658 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1bd4e59-f109-43af-8a6f-ac865b3f8843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:48.779163 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-c1bd4e59-f109-43af-8a6f-ac865b3f8843 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:48.780377 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c1bd4e59-f109-43af-8a6f-ac865b3f8843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:48.785671 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:48.842847 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1428/2833] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:48.852484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:48.853895 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:48.911735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e9bf9-6dc9-46f4-90b1-252b20954448 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:48.915444 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18485f05-cbc1-427d-aea8-d4a4a2f52a2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:48.915981 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485f05-cbc1-427d-aea8-d4a4a2f52a2b service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:48.916256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485f05-cbc1-427d-aea8-d4a4a2f52a2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:48.916524 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18485f05-cbc1-427d-aea8-d4a4a2f52a2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:48.926227 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18485f05-cbc1-427d-aea8-d4a4a2f52a2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:48.927647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18485f05-cbc1-427d-aea8-d4a4a2f52a2b service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:48.937928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1406/2834] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:54:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:54:49.201414 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd4e59-f109-43af-8a6f-ac865b3f8843 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:49.202810 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-c1bd4e59-f109-43af-8a6f-ac865b3f8843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:49.216068 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1bd4e59-f109-43af-8a6f-ac865b3f8843 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:49.220750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:49.221405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:49.221681 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:49.221955 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] PATH_INFO: `/v3/projects/c34610acfe9b4072adfae6bbdcdf88c8` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:49.223405 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c34610acfe9b4072adfae6bbdcdf88c8)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:49.225232 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:49.271812 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c34610acfe9b4072adfae6bbdcdf88c8', 'request_id': 'req-2a8afb7d-0429-450b-811e-e5c012aff69d'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.279469 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c34610acfe9b4072adfae6bbdcdf88c8', 'request_id': 'req-2a8afb7d-0429-450b-811e-e5c012aff69d'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.285327 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c34610acfe9b4072adfae6bbdcdf88c8', 'request_id': 'req-2a8afb7d-0429-450b-811e-e5c012aff69d'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.297370 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] The token cache is being invalidate because project c34610acfe9b4072adfae6bbdcdf88c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:54:49.297651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.298353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.299811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1429/2835] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:54:49 2022] DELETE /identity/v3/projects/c34610acfe9b4072adfae6bbdcdf88c8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:49.303881 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18485f05-cbc1-427d-aea8-d4a4a2f52a2b service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:49.310353 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:49.314180 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18485f05-cbc1-427d-aea8-d4a4a2f52a2b service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:49.370486 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9264504-3c3f-4418-8b9c-79a8b7dc5822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:49.371174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9264504-3c3f-4418-8b9c-79a8b7dc5822 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:49.371370 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9264504-3c3f-4418-8b9c-79a8b7dc5822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:49.371638 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9264504-3c3f-4418-8b9c-79a8b7dc5822 admin admin] PATH_INFO: `/v3/users/00357c9f83784103ae8b138c23627eef` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:49.372745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9264504-3c3f-4418-8b9c-79a8b7dc5822 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00357c9f83784103ae8b138c23627eef)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:49.374662 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9264504-3c3f-4418-8b9c-79a8b7dc5822 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:49.421618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a9264504-3c3f-4418-8b9c-79a8b7dc5822 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00357c9f83784103ae8b138c23627eef'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.422360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a9264504-3c3f-4418-8b9c-79a8b7dc5822 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00357c9f83784103ae8b138c23627eef'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.430144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a9264504-3c3f-4418-8b9c-79a8b7dc5822 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00357c9f83784103ae8b138c23627eef'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.437469 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a9264504-3c3f-4418-8b9c-79a8b7dc5822 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00357c9f83784103ae8b138c23627eef'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.449425 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-a9264504-3c3f-4418-8b9c-79a8b7dc5822 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00357c9f83784103ae8b138c23627eef'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.455120 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1407/2836] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:54:49 2022] DELETE /identity/v3/users/00357c9f83784103ae8b138c23627eef => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:49.723581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:49.725105 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:49.739017 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8afb7d-0429-450b-811e-e5c012aff69d admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:49.797176 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:49.797848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:49.798062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:49.798329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] PATH_INFO: `/v3/projects/07953e276bf842bbb8ffa662fe1d3104` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:49.799445 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=07953e276bf842bbb8ffa662fe1d3104)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:49.801364 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:49.838677 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:49.848689 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07953e276bf842bbb8ffa662fe1d3104', 'request_id': 'req-41c11ef8-8107-44e1-8cc1-99a3b9498642'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.855692 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07953e276bf842bbb8ffa662fe1d3104', 'request_id': 'req-41c11ef8-8107-44e1-8cc1-99a3b9498642'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.861093 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07953e276bf842bbb8ffa662fe1d3104', 'request_id': 'req-41c11ef8-8107-44e1-8cc1-99a3b9498642'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.871796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] The token cache is being invalidate because project 07953e276bf842bbb8ffa662fe1d3104 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:54:49.872147 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.872701 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-41c11ef8-8107-44e1-8cc1-99a3b9498642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:54:49.874185 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1430/2837] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:54:49 2022] DELETE /identity/v3/projects/07953e276bf842bbb8ffa662fe1d3104 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:49.879723 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de86390-7879-4c5f-901e-81a35406d996 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:49.880195 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de86390-7879-4c5f-901e-81a35406d996 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:49.880309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de86390-7879-4c5f-901e-81a35406d996 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:49.900832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-6de86390-7879-4c5f-901e-81a35406d996 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:49.903121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-6de86390-7879-4c5f-901e-81a35406d996 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:49.961996 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1408/2838] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:49.972119 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025c85d-3cdc-44a2-9b75-318e2d04899d None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:49.972344 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025c85d-3cdc-44a2-9b75-318e2d04899d None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:49.972621 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2025c85d-3cdc-44a2-9b75-318e2d04899d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:49.992103 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2025c85d-3cdc-44a2-9b75-318e2d04899d None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:49.993340 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2025c85d-3cdc-44a2-9b75-318e2d04899d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:50.001329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1431/2839] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:50.005307 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de86390-7879-4c5f-901e-81a35406d996 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.015481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de86390-7879-4c5f-901e-81a35406d996 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.019917 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c4e9c-e439-4eac-8524-36d12b7b12a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.020504 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c4e9c-e439-4eac-8524-36d12b7b12a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.020791 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c4e9c-e439-4eac-8524-36d12b7b12a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.021061 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c4e9c-e439-4eac-8524-36d12b7b12a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.021690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c4e9c-e439-4eac-8524-36d12b7b12a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:54:50.021985 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c4e9c-e439-4eac-8524-36d12b7b12a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.023546 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c4e9c-e439-4eac-8524-36d12b7b12a2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.035803 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1409/2840] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:54:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:54:50.039959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025c85d-3cdc-44a2-9b75-318e2d04899d None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.049054 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2025c85d-3cdc-44a2-9b75-318e2d04899d None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.053544 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45af87-86ed-404a-a10c-9729037f3436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.054123 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45af87-86ed-404a-a10c-9729037f3436 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.054392 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45af87-86ed-404a-a10c-9729037f3436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.054656 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c45af87-86ed-404a-a10c-9729037f3436 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.055272 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45af87-86ed-404a-a10c-9729037f3436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.057166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c45af87-86ed-404a-a10c-9729037f3436 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.084645 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1432/2841] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:54:50 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:54:50.088747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c4e9c-e439-4eac-8524-36d12b7b12a2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.099024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c4e9c-e439-4eac-8524-36d12b7b12a2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.108894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:50.160281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691842c-38a5-4f42-b8d0-3b7f51b12041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.160907 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691842c-38a5-4f42-b8d0-3b7f51b12041 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.161185 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691842c-38a5-4f42-b8d0-3b7f51b12041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.161454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691842c-38a5-4f42-b8d0-3b7f51b12041 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.162063 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691842c-38a5-4f42-b8d0-3b7f51b12041 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.163947 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b691842c-38a5-4f42-b8d0-3b7f51b12041 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.184217 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1410/2842] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:54:50 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:54:50.188319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45af87-86ed-404a-a10c-9729037f3436 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.198454 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c45af87-86ed-404a-a10c-9729037f3436 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.203697 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d69884-7a84-4f0e-a806-9c0592302f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.204285 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d69884-7a84-4f0e-a806-9c0592302f32 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.204561 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d69884-7a84-4f0e-a806-9c0592302f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.204868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d69884-7a84-4f0e-a806-9c0592302f32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.205501 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d69884-7a84-4f0e-a806-9c0592302f32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.207065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d69884-7a84-4f0e-a806-9c0592302f32 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.217160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1433/2843] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:54:50.221005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691842c-38a5-4f42-b8d0-3b7f51b12041 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.230824 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691842c-38a5-4f42-b8d0-3b7f51b12041 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.235336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82d2497-2d94-4f16-8d9c-567e61a386cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.235916 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d2497-2d94-4f16-8d9c-567e61a386cf admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.236191 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d2497-2d94-4f16-8d9c-567e61a386cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.236459 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82d2497-2d94-4f16-8d9c-567e61a386cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.237114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d2497-2d94-4f16-8d9c-567e61a386cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.238675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82d2497-2d94-4f16-8d9c-567e61a386cf admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.248782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1411/2844] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:54:50.252699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d69884-7a84-4f0e-a806-9c0592302f32 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.263087 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d69884-7a84-4f0e-a806-9c0592302f32 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.267881 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59289e6-e740-41cc-a501-7ce151262d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.268518 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59289e6-e740-41cc-a501-7ce151262d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.268811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59289e6-e740-41cc-a501-7ce151262d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.269082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59289e6-e740-41cc-a501-7ce151262d5d admin admin] PATH_INFO: `/v3/projects/6ae90c71dab74642aefb85aa51932651/users/63ffee5587144e4dbcf6e756ed4e35d9/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.283516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59289e6-e740-41cc-a501-7ce151262d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ae90c71dab74642aefb85aa51932651, user_id=63ffee5587144e4dbcf6e756ed4e35d9, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.286156 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59289e6-e740-41cc-a501-7ce151262d5d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.299244 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1434/2845] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:54:50 2022] PUT /identity/v3/projects/6ae90c71dab74642aefb85aa51932651/users/63ffee5587144e4dbcf6e756ed4e35d9/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:50.360944 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91474e9-a74f-4f31-a19e-33367ab15448 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.361224 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91474e9-a74f-4f31-a19e-33367ab15448 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.361493 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91474e9-a74f-4f31-a19e-33367ab15448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.392114 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-b91474e9-a74f-4f31-a19e-33367ab15448 None None] MFA Rules not processed for user `63ffee5587144e4dbcf6e756ed4e35d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:50.393345 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b91474e9-a74f-4f31-a19e-33367ab15448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:50.458520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1412/2846] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:50.463038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59289e6-e740-41cc-a501-7ce151262d5d admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.525672 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59289e6-e740-41cc-a501-7ce151262d5d admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.530943 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a3109-0a9e-4371-8d8a-2073f7b7a88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.531789 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a3109-0a9e-4371-8d8a-2073f7b7a88e admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.532064 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a3109-0a9e-4371-8d8a-2073f7b7a88e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.532334 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77a3109-0a9e-4371-8d8a-2073f7b7a88e admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.532978 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a3109-0a9e-4371-8d8a-2073f7b7a88e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.534953 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77a3109-0a9e-4371-8d8a-2073f7b7a88e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.563306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1435/2847] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:54:50 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:54:50.567288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91474e9-a74f-4f31-a19e-33367ab15448 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.578461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91474e9-a74f-4f31-a19e-33367ab15448 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.633621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da659b96-a554-4255-a8ea-ef778f08a035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.634230 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da659b96-a554-4255-a8ea-ef778f08a035 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.634509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da659b96-a554-4255-a8ea-ef778f08a035 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.634782 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da659b96-a554-4255-a8ea-ef778f08a035 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.635416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da659b96-a554-4255-a8ea-ef778f08a035 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.637372 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da659b96-a554-4255-a8ea-ef778f08a035 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.658830 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1413/2848] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:54:50 2022] POST /identity/v3/users => generated 299 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:54:50.662570 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a3109-0a9e-4371-8d8a-2073f7b7a88e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.673565 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77a3109-0a9e-4371-8d8a-2073f7b7a88e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.678094 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec0d82-4ad3-4385-bb33-9d0988323acb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.678684 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec0d82-4ad3-4385-bb33-9d0988323acb admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.678960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec0d82-4ad3-4385-bb33-9d0988323acb admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.679226 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec0d82-4ad3-4385-bb33-9d0988323acb admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.679852 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec0d82-4ad3-4385-bb33-9d0988323acb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.681460 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec0d82-4ad3-4385-bb33-9d0988323acb admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.692131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1436/2849] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:54:50.696012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da659b96-a554-4255-a8ea-ef778f08a035 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.706657 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da659b96-a554-4255-a8ea-ef778f08a035 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.711279 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3ac66-7062-43fb-8a85-6c8d26fe65d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.711912 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3ac66-7062-43fb-8a85-6c8d26fe65d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.712211 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3ac66-7062-43fb-8a85-6c8d26fe65d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.712477 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed3ac66-7062-43fb-8a85-6c8d26fe65d9 admin admin] PATH_INFO: `/v3/projects/52847548cea44c6481d32ec97d579c93/users/af5208dbabdb4ceeb0e666410c3292f1/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.716103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:50.729084 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3ac66-7062-43fb-8a85-6c8d26fe65d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52847548cea44c6481d32ec97d579c93, user_id=af5208dbabdb4ceeb0e666410c3292f1, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.732106 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed3ac66-7062-43fb-8a85-6c8d26fe65d9 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.744506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1414/2850] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:54:50 2022] PUT /identity/v3/projects/52847548cea44c6481d32ec97d579c93/users/af5208dbabdb4ceeb0e666410c3292f1/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:54:50.748546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec0d82-4ad3-4385-bb33-9d0988323acb admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.758491 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec0d82-4ad3-4385-bb33-9d0988323acb admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.791053 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:54:50.818015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddaeddf8-e892-424e-8ce9-8961ffb459f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.818617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaeddf8-e892-424e-8ce9-8961ffb459f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.818885 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaeddf8-e892-424e-8ce9-8961ffb459f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.819154 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddaeddf8-e892-424e-8ce9-8961ffb459f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.819783 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaeddf8-e892-424e-8ce9-8961ffb459f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.821394 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddaeddf8-e892-424e-8ce9-8961ffb459f9 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.831506 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1437/2851] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:54:50 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:54:50.835688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3ac66-7062-43fb-8a85-6c8d26fe65d9 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.850308 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed3ac66-7062-43fb-8a85-6c8d26fe65d9 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:50.855461 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322292ac-5f04-4e94-ab98-12984885634c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:50.856357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322292ac-5f04-4e94-ab98-12984885634c admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.856631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322292ac-5f04-4e94-ab98-12984885634c admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.856937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322292ac-5f04-4e94-ab98-12984885634c admin admin] PATH_INFO: `/v3/projects/52847548cea44c6481d32ec97d579c93/users/af5208dbabdb4ceeb0e666410c3292f1/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.858858 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322292ac-5f04-4e94-ab98-12984885634c admin admin] RBAC: Authorizing `identity:create_grant(project_id=52847548cea44c6481d32ec97d579c93, user_id=af5208dbabdb4ceeb0e666410c3292f1, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:50.861548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322292ac-5f04-4e94-ab98-12984885634c admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:50.872484 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1415/2852] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:54:50 2022] PUT /identity/v3/projects/52847548cea44c6481d32ec97d579c93/users/af5208dbabdb4ceeb0e666410c3292f1/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:54:50.882512 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c818c76-5262-4604-aa06-eb8cf6d18e5c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:50.882799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c818c76-5262-4604-aa06-eb8cf6d18e5c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:50.883098 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c818c76-5262-4604-aa06-eb8cf6d18e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:50.911616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-9c818c76-5262-4604-aa06-eb8cf6d18e5c None None] MFA Rules not processed for user `af5208dbabdb4ceeb0e666410c3292f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:54:50.912857 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9c818c76-5262-4604-aa06-eb8cf6d18e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:50.979254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1438/2853] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:54:50 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:54:50.989997 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322292ac-5f04-4e94-ab98-12984885634c admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:50.991668 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-322292ac-5f04-4e94-ab98-12984885634c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:51.049013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322292ac-5f04-4e94-ab98-12984885634c admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:51.052531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9044d5c-ab09-4227-8200-7ce65c16ebe4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:51.053146 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044d5c-ab09-4227-8200-7ce65c16ebe4 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:51.053423 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044d5c-ab09-4227-8200-7ce65c16ebe4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:51.053690 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9044d5c-ab09-4227-8200-7ce65c16ebe4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:51.110027 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9044d5c-ab09-4227-8200-7ce65c16ebe4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:51.111448 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9044d5c-ab09-4227-8200-7ce65c16ebe4 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:51.120283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1416/2854] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:54:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:54:55.497134 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c818c76-5262-4604-aa06-eb8cf6d18e5c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:54:55.498516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9c818c76-5262-4604-aa06-eb8cf6d18e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:54:55.510286 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c818c76-5262-4604-aa06-eb8cf6d18e5c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:54:55.514554 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294dbc82-e40f-410c-9d0e-6c0c40efa22e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:54:55.515165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dbc82-e40f-410c-9d0e-6c0c40efa22e service cinder] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:54:55.515436 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dbc82-e40f-410c-9d0e-6c0c40efa22e service cinder] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:54:55.515698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294dbc82-e40f-410c-9d0e-6c0c40efa22e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:54:55.524371 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294dbc82-e40f-410c-9d0e-6c0c40efa22e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:54:55.525786 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294dbc82-e40f-410c-9d0e-6c0c40efa22e service cinder] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:54:55.535423 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1439/2855] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:54:55 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:55:00.437419 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9044d5c-ab09-4227-8200-7ce65c16ebe4 service cinder] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:00.498660 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9044d5c-ab09-4227-8200-7ce65c16ebe4 service cinder] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:00.503151 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:00.503772 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:00.504048 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:00.504319 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] PATH_INFO: `/v3/users/63ffee5587144e4dbcf6e756ed4e35d9` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:00.505431 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ffee5587144e4dbcf6e756ed4e35d9)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:00.507336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:00.535174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:00.553952 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ffee5587144e4dbcf6e756ed4e35d9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:00.554590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ffee5587144e4dbcf6e756ed4e35d9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:00.562337 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ffee5587144e4dbcf6e756ed4e35d9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:00.569537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ffee5587144e4dbcf6e756ed4e35d9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:00.582622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ffee5587144e4dbcf6e756ed4e35d9'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:00.588368 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1417/2856] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:55:00 2022] DELETE /identity/v3/users/63ffee5587144e4dbcf6e756ed4e35d9 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:00.592992 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228d9ae-9e02-4c4d-b8f4-6392929d18e5 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:00.593438 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228d9ae-9e02-4c4d-b8f4-6392929d18e5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:00.593559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228d9ae-9e02-4c4d-b8f4-6392929d18e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:00.611910 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d228d9ae-9e02-4c4d-b8f4-6392929d18e5 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:00.613120 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d228d9ae-9e02-4c4d-b8f4-6392929d18e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:00.670828 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1440/2857] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:00.680340 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:00.681801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:00.738512 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4519c398-5e1f-49e3-922e-0ae979d997ba admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:00.742137 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde74ac0-a00f-4f99-8f66-f176f5b051d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:00.742731 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde74ac0-a00f-4f99-8f66-f176f5b051d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:00.743003 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde74ac0-a00f-4f99-8f66-f176f5b051d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:00.743268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde74ac0-a00f-4f99-8f66-f176f5b051d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:00.752628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde74ac0-a00f-4f99-8f66-f176f5b051d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:00.754069 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde74ac0-a00f-4f99-8f66-f176f5b051d1 service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:00.763967 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1418/2858] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:55:01.029799 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228d9ae-9e02-4c4d-b8f4-6392929d18e5 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:01.031201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d228d9ae-9e02-4c4d-b8f4-6392929d18e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:01.045938 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228d9ae-9e02-4c4d-b8f4-6392929d18e5 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:01.050525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:01.051165 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:01.052031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:01.052303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] PATH_INFO: `/v3/projects/6ae90c71dab74642aefb85aa51932651` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:01.053615 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ae90c71dab74642aefb85aa51932651)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:01.055502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:01.095377 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:01.104194 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ae90c71dab74642aefb85aa51932651', 'request_id': 'req-23de9aa0-255e-4e4f-bb61-7b75a2e88370'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.112409 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ae90c71dab74642aefb85aa51932651', 'request_id': 'req-23de9aa0-255e-4e4f-bb61-7b75a2e88370'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.117842 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ae90c71dab74642aefb85aa51932651', 'request_id': 'req-23de9aa0-255e-4e4f-bb61-7b75a2e88370'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.129257 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] The token cache is being invalidate because project 6ae90c71dab74642aefb85aa51932651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:55:01.129539 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.130160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.131610 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1441/2859] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:55:01 2022] DELETE /identity/v3/projects/6ae90c71dab74642aefb85aa51932651 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:01.135651 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde74ac0-a00f-4f99-8f66-f176f5b051d1 service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:01.145343 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde74ac0-a00f-4f99-8f66-f176f5b051d1 service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:01.201017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3ee736-772d-4546-9103-3f535a90b005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:01.201628 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ee736-772d-4546-9103-3f535a90b005 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:01.201901 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ee736-772d-4546-9103-3f535a90b005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:01.202167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3ee736-772d-4546-9103-3f535a90b005 admin admin] PATH_INFO: `/v3/users/af5208dbabdb4ceeb0e666410c3292f1` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:01.203274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ee736-772d-4546-9103-3f535a90b005 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af5208dbabdb4ceeb0e666410c3292f1)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:01.205242 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3ee736-772d-4546-9103-3f535a90b005 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:01.249956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9e3ee736-772d-4546-9103-3f535a90b005 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af5208dbabdb4ceeb0e666410c3292f1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.250578 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9e3ee736-772d-4546-9103-3f535a90b005 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af5208dbabdb4ceeb0e666410c3292f1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.253012 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:01.258530 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9e3ee736-772d-4546-9103-3f535a90b005 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af5208dbabdb4ceeb0e666410c3292f1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.266761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9e3ee736-772d-4546-9103-3f535a90b005 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af5208dbabdb4ceeb0e666410c3292f1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.279174 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-9e3ee736-772d-4546-9103-3f535a90b005 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af5208dbabdb4ceeb0e666410c3292f1'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.285076 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1419/2860] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:55:01 2022] DELETE /identity/v3/users/af5208dbabdb4ceeb0e666410c3292f1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:01.550424 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:01.551949 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:01.566811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23de9aa0-255e-4e4f-bb61-7b75a2e88370 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:01.624526 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:01.625239 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:01.625450 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:01.625716 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] PATH_INFO: `/v3/projects/52847548cea44c6481d32ec97d579c93` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:01.626811 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=52847548cea44c6481d32ec97d579c93)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:01.628713 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:01.673886 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52847548cea44c6481d32ec97d579c93', 'request_id': 'req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.682050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52847548cea44c6481d32ec97d579c93', 'request_id': 'req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.687659 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52847548cea44c6481d32ec97d579c93', 'request_id': 'req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.698901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] The token cache is being invalidate because project 52847548cea44c6481d32ec97d579c93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:55:01.699169 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.699698 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-20f4061f-3580-4dc4-9a36-8ca7a3d83e2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:01.701202 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1442/2861] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:55:01 2022] DELETE /identity/v3/projects/52847548cea44c6481d32ec97d579c93 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:01.707034 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa05c7-0091-4f10-866d-010fdd6c34b3 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:01.707517 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa05c7-0091-4f10-866d-010fdd6c34b3 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:01.707625 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76aa05c7-0091-4f10-866d-010fdd6c34b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:01.729853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-76aa05c7-0091-4f10-866d-010fdd6c34b3 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:01.730960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-76aa05c7-0091-4f10-866d-010fdd6c34b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:01.788888 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1420/2862] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:01.798743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c29af-f07f-4153-95ac-b76e8df4a949 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:01.799020 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c29af-f07f-4153-95ac-b76e8df4a949 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:01.799290 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c29af-f07f-4153-95ac-b76e8df4a949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:01.818715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-175c29af-f07f-4153-95ac-b76e8df4a949 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:01.819925 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-175c29af-f07f-4153-95ac-b76e8df4a949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:01.828311 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1443/2863] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:01.832586 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa05c7-0091-4f10-866d-010fdd6c34b3 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:01.843132 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76aa05c7-0091-4f10-866d-010fdd6c34b3 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:01.853698 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc3ce-30de-4348-be3d-e1e5307fa669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:01.854306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc3ce-30de-4348-be3d-e1e5307fa669 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:01.854621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc3ce-30de-4348-be3d-e1e5307fa669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:01.854990 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fdc3ce-30de-4348-be3d-e1e5307fa669 admin admin] PATH_INFO: `/v3/domains` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:01.855667 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdc3ce-30de-4348-be3d-e1e5307fa669 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81665) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:55:01.855968 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdc3ce-30de-4348-be3d-e1e5307fa669 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:01.857632 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fdc3ce-30de-4348-be3d-e1e5307fa669 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:01.869410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1421/2864] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:55:01.874052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c29af-f07f-4153-95ac-b76e8df4a949 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:01.883275 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c29af-f07f-4153-95ac-b76e8df4a949 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:01.888573 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d49482-6766-4782-a4a0-3fc2b8a1fa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:01.889292 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d49482-6766-4782-a4a0-3fc2b8a1fa90 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:01.889566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d49482-6766-4782-a4a0-3fc2b8a1fa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:01.889832 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d49482-6766-4782-a4a0-3fc2b8a1fa90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:01.890460 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d49482-6766-4782-a4a0-3fc2b8a1fa90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:01.892353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d49482-6766-4782-a4a0-3fc2b8a1fa90 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:01.919801 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1444/2865] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:55:01 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:55:01.924126 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc3ce-30de-4348-be3d-e1e5307fa669 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:01.934752 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fdc3ce-30de-4348-be3d-e1e5307fa669 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:01.990218 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb88422-b9fe-4db6-90bf-f0693b1a3ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:01.990779 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb88422-b9fe-4db6-90bf-f0693b1a3ecd admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:01.991056 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb88422-b9fe-4db6-90bf-f0693b1a3ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:01.991328 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb88422-b9fe-4db6-90bf-f0693b1a3ecd admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:01.991960 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb88422-b9fe-4db6-90bf-f0693b1a3ecd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:01.993948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb88422-b9fe-4db6-90bf-f0693b1a3ecd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:02.015462 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1422/2866] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:55:01 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:55:02.019583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d49482-6766-4782-a4a0-3fc2b8a1fa90 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:02.030559 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d49482-6766-4782-a4a0-3fc2b8a1fa90 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:02.035433 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baafa14-ed85-4dc4-9a78-9d504edb267a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:02.036028 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafa14-ed85-4dc4-9a78-9d504edb267a admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.036307 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafa14-ed85-4dc4-9a78-9d504edb267a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.036581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baafa14-ed85-4dc4-9a78-9d504edb267a admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.037279 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baafa14-ed85-4dc4-9a78-9d504edb267a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:02.038850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baafa14-ed85-4dc4-9a78-9d504edb267a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:02.049796 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1445/2867] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:55:02.053647 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb88422-b9fe-4db6-90bf-f0693b1a3ecd admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:02.064132 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb88422-b9fe-4db6-90bf-f0693b1a3ecd admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:02.068747 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3b3d5-c270-4eed-9828-a8f8506636d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:02.069359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3b3d5-c270-4eed-9828-a8f8506636d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.069636 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3b3d5-c270-4eed-9828-a8f8506636d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.069903 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3b3d5-c270-4eed-9828-a8f8506636d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.070539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3b3d5-c270-4eed-9828-a8f8506636d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:02.072109 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3b3d5-c270-4eed-9828-a8f8506636d5 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:02.083392 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1423/2868] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:55:02.087433 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baafa14-ed85-4dc4-9a78-9d504edb267a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:02.097975 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baafa14-ed85-4dc4-9a78-9d504edb267a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:02.103604 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffd495-5283-4c04-8883-40a2b143297d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:02.104247 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffd495-5283-4c04-8883-40a2b143297d admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.105275 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffd495-5283-4c04-8883-40a2b143297d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.105630 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ffd495-5283-4c04-8883-40a2b143297d admin admin] PATH_INFO: `/v3/projects/cfe9b63bc99f4ec29cbac58cbea49ba7/users/35e86c0d7c0c4437942929dbf63bc2bd/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.121124 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffd495-5283-4c04-8883-40a2b143297d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfe9b63bc99f4ec29cbac58cbea49ba7, user_id=35e86c0d7c0c4437942929dbf63bc2bd, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:02.123735 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ffd495-5283-4c04-8883-40a2b143297d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:02.125962 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:02.137542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1446/2869] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:55:02 2022] PUT /identity/v3/projects/cfe9b63bc99f4ec29cbac58cbea49ba7/users/35e86c0d7c0c4437942929dbf63bc2bd/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:55:02.148593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802f1b-f9f1-4de0-95f6-c69ded584657 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.148904 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802f1b-f9f1-4de0-95f6-c69ded584657 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.149170 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802f1b-f9f1-4de0-95f6-c69ded584657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.177325 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-e7802f1b-f9f1-4de0-95f6-c69ded584657 None None] MFA Rules not processed for user `35e86c0d7c0c4437942929dbf63bc2bd`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:02.178506 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-e7802f1b-f9f1-4de0-95f6-c69ded584657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:02.197557 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:02.243004 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1424/2870] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:02.247902 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffd495-5283-4c04-8883-40a2b143297d admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:02.309530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ffd495-5283-4c04-8883-40a2b143297d admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:02.314166 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa47f1-f606-42a3-a3d0-dc6ba676fb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:02.314761 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa47f1-f606-42a3-a3d0-dc6ba676fb2d admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.315062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa47f1-f606-42a3-a3d0-dc6ba676fb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.315312 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6aa47f1-f606-42a3-a3d0-dc6ba676fb2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.315939 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa47f1-f606-42a3-a3d0-dc6ba676fb2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:02.317851 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6aa47f1-f606-42a3-a3d0-dc6ba676fb2d admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:02.344577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1447/2871] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:55:02 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:55:02.348777 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7802f1b-f9f1-4de0-95f6-c69ded584657 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:02.359177 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7802f1b-f9f1-4de0-95f6-c69ded584657 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:02.415445 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de5fbe-4bdb-4625-a559-793c120399a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:02.416053 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de5fbe-4bdb-4625-a559-793c120399a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.416336 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de5fbe-4bdb-4625-a559-793c120399a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.416611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5de5fbe-4bdb-4625-a559-793c120399a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.417965 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de5fbe-4bdb-4625-a559-793c120399a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:02.419872 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5de5fbe-4bdb-4625-a559-793c120399a0 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:02.440675 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1425/2872] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:55:02 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:55:02.444848 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa47f1-f606-42a3-a3d0-dc6ba676fb2d admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:02.455777 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6aa47f1-f606-42a3-a3d0-dc6ba676fb2d admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:02.461259 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a428ee6-4025-4ac9-8088-6c0aacb45699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:02.461859 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a428ee6-4025-4ac9-8088-6c0aacb45699 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.462158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a428ee6-4025-4ac9-8088-6c0aacb45699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.462432 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a428ee6-4025-4ac9-8088-6c0aacb45699 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.463065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a428ee6-4025-4ac9-8088-6c0aacb45699 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:02.464967 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a428ee6-4025-4ac9-8088-6c0aacb45699 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:02.475888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1448/2873] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:55:02.480539 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de5fbe-4bdb-4625-a559-793c120399a0 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:02.490427 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5de5fbe-4bdb-4625-a559-793c120399a0 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:02.495730 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16addc1-9109-42a0-bd35-f5eb4bd7fec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:02.496359 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16addc1-9109-42a0-bd35-f5eb4bd7fec2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.496641 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16addc1-9109-42a0-bd35-f5eb4bd7fec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.496948 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16addc1-9109-42a0-bd35-f5eb4bd7fec2 admin admin] PATH_INFO: `/v3/projects/165761b78f334813b22793f7fd2d6292/users/4d83bd8e8450468e8394be09c8571261/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.511784 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16addc1-9109-42a0-bd35-f5eb4bd7fec2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=165761b78f334813b22793f7fd2d6292, user_id=4d83bd8e8450468e8394be09c8571261, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:02.514548 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16addc1-9109-42a0-bd35-f5eb4bd7fec2 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:02.526520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1426/2874] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:55:02 2022] PUT /identity/v3/projects/165761b78f334813b22793f7fd2d6292/users/4d83bd8e8450468e8394be09c8571261/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:02.530389 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a428ee6-4025-4ac9-8088-6c0aacb45699 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:02.540280 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a428ee6-4025-4ac9-8088-6c0aacb45699 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:02.596059 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70810be-e7a4-41ff-a0aa-183ed419a9c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:02.596660 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70810be-e7a4-41ff-a0aa-183ed419a9c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.596976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70810be-e7a4-41ff-a0aa-183ed419a9c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.597252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70810be-e7a4-41ff-a0aa-183ed419a9c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.597878 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70810be-e7a4-41ff-a0aa-183ed419a9c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:02.599451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70810be-e7a4-41ff-a0aa-183ed419a9c7 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:02.609596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1449/2875] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:55:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:55:02.613956 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16addc1-9109-42a0-bd35-f5eb4bd7fec2 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:02.624226 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16addc1-9109-42a0-bd35-f5eb4bd7fec2 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:02.628959 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c789384-a73f-4717-bd0d-740ff2f614cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:02.629753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c789384-a73f-4717-bd0d-740ff2f614cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.630037 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c789384-a73f-4717-bd0d-740ff2f614cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.630309 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c789384-a73f-4717-bd0d-740ff2f614cb admin admin] PATH_INFO: `/v3/projects/165761b78f334813b22793f7fd2d6292/users/4d83bd8e8450468e8394be09c8571261/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.632264 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c789384-a73f-4717-bd0d-740ff2f614cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=165761b78f334813b22793f7fd2d6292, user_id=4d83bd8e8450468e8394be09c8571261, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:02.634934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c789384-a73f-4717-bd0d-740ff2f614cb admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:02.645704 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1427/2876] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:55:02 2022] PUT /identity/v3/projects/165761b78f334813b22793f7fd2d6292/users/4d83bd8e8450468e8394be09c8571261/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:55:02.657459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8befea2-6675-4517-aa40-19f18303dc34 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.657976 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8befea2-6675-4517-aa40-19f18303dc34 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.658089 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8befea2-6675-4517-aa40-19f18303dc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.685326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-d8befea2-6675-4517-aa40-19f18303dc34 None None] MFA Rules not processed for user `4d83bd8e8450468e8394be09c8571261`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:02.686546 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d8befea2-6675-4517-aa40-19f18303dc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:02.736428 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:02.755393 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1450/2877] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:02 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:02.766734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c789384-a73f-4717-bd0d-740ff2f614cb admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:02.768485 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-5c789384-a73f-4717-bd0d-740ff2f614cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:02.834273 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c789384-a73f-4717-bd0d-740ff2f614cb admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:02.838001 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e273fb9f-c427-46ec-adff-277dcde161e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:02.838603 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273fb9f-c427-46ec-adff-277dcde161e3 service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:02.838890 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273fb9f-c427-46ec-adff-277dcde161e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:02.839167 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e273fb9f-c427-46ec-adff-277dcde161e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:02.867356 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:02.896122 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273fb9f-c427-46ec-adff-277dcde161e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:02.897590 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e273fb9f-c427-46ec-adff-277dcde161e3 service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:02.906593 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1428/2878] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:02 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:55:05.478275 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8befea2-6675-4517-aa40-19f18303dc34 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:05.538433 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8befea2-6675-4517-aa40-19f18303dc34 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:05.543468 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:05.544015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:05.544289 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:05.544558 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] PATH_INFO: `/v3/users/35e86c0d7c0c4437942929dbf63bc2bd` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:05.545676 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] RBAC: Authorizing `identity:delete_user(user_id=35e86c0d7c0c4437942929dbf63bc2bd)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:05.547540 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:05.595452 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '35e86c0d7c0c4437942929dbf63bc2bd'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:05.603049 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '35e86c0d7c0c4437942929dbf63bc2bd'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:05.603751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '35e86c0d7c0c4437942929dbf63bc2bd'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:05.617062 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '35e86c0d7c0c4437942929dbf63bc2bd'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:05.624834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '35e86c0d7c0c4437942929dbf63bc2bd'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:05.630900 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1451/2879] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:55:05 2022] DELETE /identity/v3/users/35e86c0d7c0c4437942929dbf63bc2bd => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:05.635332 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670e29e-1594-4aec-a0bb-ed0bb1137517 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:05.635947 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670e29e-1594-4aec-a0bb-ed0bb1137517 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:05.636436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4670e29e-1594-4aec-a0bb-ed0bb1137517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:05.655262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4670e29e-1594-4aec-a0bb-ed0bb1137517 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:05.656701 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4670e29e-1594-4aec-a0bb-ed0bb1137517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:05.720928 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1429/2880] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:05.731161 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:05.732651 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:05.788827 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9748cd82-b548-476f-aded-025d5479c9df admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:05.793255 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7b416-3ed3-499d-b382-5841361fda00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:05.793900 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7b416-3ed3-499d-b382-5841361fda00 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:05.794125 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7b416-3ed3-499d-b382-5841361fda00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:05.794399 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e7b416-3ed3-499d-b382-5841361fda00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:05.804234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7b416-3ed3-499d-b382-5841361fda00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:05.805715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e7b416-3ed3-499d-b382-5841361fda00 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:05.815490 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1452/2881] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 06 22:55:06.078287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670e29e-1594-4aec-a0bb-ed0bb1137517 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:06.079905 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4670e29e-1594-4aec-a0bb-ed0bb1137517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:06.095455 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4670e29e-1594-4aec-a0bb-ed0bb1137517 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:06.101623 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:06.102447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:06.102801 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:06.103058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] PATH_INFO: `/v3/projects/cfe9b63bc99f4ec29cbac58cbea49ba7` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:06.105101 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfe9b63bc99f4ec29cbac58cbea49ba7)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:06.108287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:06.155428 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfe9b63bc99f4ec29cbac58cbea49ba7', 'request_id': 'req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.161674 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfe9b63bc99f4ec29cbac58cbea49ba7', 'request_id': 'req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.169233 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfe9b63bc99f4ec29cbac58cbea49ba7', 'request_id': 'req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.179142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] The token cache is being invalidate because project cfe9b63bc99f4ec29cbac58cbea49ba7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:55:06.179421 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.180018 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.181509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1430/2882] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:55:06 2022] DELETE /identity/v3/projects/cfe9b63bc99f4ec29cbac58cbea49ba7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:06.185574 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7b416-3ed3-499d-b382-5841361fda00 service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:06.195201 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e7b416-3ed3-499d-b382-5841361fda00 service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:06.203353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:06.253605 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9656c4b3-339f-489f-9289-c36699852fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:06.254288 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656c4b3-339f-489f-9289-c36699852fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:06.254566 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656c4b3-339f-489f-9289-c36699852fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:06.254839 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9656c4b3-339f-489f-9289-c36699852fa5 admin admin] PATH_INFO: `/v3/users/4d83bd8e8450468e8394be09c8571261` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:06.255954 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656c4b3-339f-489f-9289-c36699852fa5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d83bd8e8450468e8394be09c8571261)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:06.257919 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9656c4b3-339f-489f-9289-c36699852fa5 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:06.302986 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9656c4b3-339f-489f-9289-c36699852fa5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d83bd8e8450468e8394be09c8571261'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.310329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9656c4b3-339f-489f-9289-c36699852fa5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d83bd8e8450468e8394be09c8571261'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.310946 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9656c4b3-339f-489f-9289-c36699852fa5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d83bd8e8450468e8394be09c8571261'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.323043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9656c4b3-339f-489f-9289-c36699852fa5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d83bd8e8450468e8394be09c8571261'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.330287 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-9656c4b3-339f-489f-9289-c36699852fa5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d83bd8e8450468e8394be09c8571261'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.336160 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1453/2883] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:55:06 2022] DELETE /identity/v3/users/4d83bd8e8450468e8394be09c8571261 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:06.605631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:06.607132 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:06.623017 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68272ac2-78f9-4432-8d1e-b6ac7d66f3aa admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:06.672991 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:06.683574 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:06.684436 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:06.684949 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:06.685228 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] PATH_INFO: `/v3/projects/165761b78f334813b22793f7fd2d6292` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:06.686374 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=165761b78f334813b22793f7fd2d6292)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:06.688274 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:06.735489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '165761b78f334813b22793f7fd2d6292', 'request_id': 'req-99bb240e-6b18-4b4d-82b6-5cdc055600c6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.741544 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '165761b78f334813b22793f7fd2d6292', 'request_id': 'req-99bb240e-6b18-4b4d-82b6-5cdc055600c6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.749620 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '165761b78f334813b22793f7fd2d6292', 'request_id': 'req-99bb240e-6b18-4b4d-82b6-5cdc055600c6'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.759911 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] The token cache is being invalidate because project 165761b78f334813b22793f7fd2d6292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:55:06.760198 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.760869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-99bb240e-6b18-4b4d-82b6-5cdc055600c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:06.762915 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1431/2884] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:55:06 2022] DELETE /identity/v3/projects/165761b78f334813b22793f7fd2d6292 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:06.767867 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742d02e-80b1-488e-81b3-481e4dc673dd None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:06.768338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742d02e-80b1-488e-81b3-481e4dc673dd None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:06.768447 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8742d02e-80b1-488e-81b3-481e4dc673dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:06.787850 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-8742d02e-80b1-488e-81b3-481e4dc673dd None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:06.789070 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-8742d02e-80b1-488e-81b3-481e4dc673dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:06.847863 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1454/2885] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:06.855634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac109c69-5c0a-49e4-bfbb-a51f01f14893 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:06.855936 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac109c69-5c0a-49e4-bfbb-a51f01f14893 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:06.856205 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac109c69-5c0a-49e4-bfbb-a51f01f14893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:06.876322 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-ac109c69-5c0a-49e4-bfbb-a51f01f14893 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:06.877567 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-ac109c69-5c0a-49e4-bfbb-a51f01f14893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:06.885440 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1432/2886] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:06.889700 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742d02e-80b1-488e-81b3-481e4dc673dd None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:06.899611 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8742d02e-80b1-488e-81b3-481e4dc673dd None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:06.904167 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dabac7-7769-4455-aac7-5113b5375b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:06.904750 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dabac7-7769-4455-aac7-5113b5375b63 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:06.905052 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dabac7-7769-4455-aac7-5113b5375b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:06.905577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7dabac7-7769-4455-aac7-5113b5375b63 admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:06.906459 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dabac7-7769-4455-aac7-5113b5375b63 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:55:06.906756 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dabac7-7769-4455-aac7-5113b5375b63 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:06.908353 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7dabac7-7769-4455-aac7-5113b5375b63 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:06.920066 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1455/2887] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 06 22:55:06.924152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac109c69-5c0a-49e4-bfbb-a51f01f14893 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:06.935040 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac109c69-5c0a-49e4-bfbb-a51f01f14893 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:06.939854 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266bdc2-b668-4d68-9bb6-c4b43df75f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:06.940446 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266bdc2-b668-4d68-9bb6-c4b43df75f36 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:06.940718 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266bdc2-b668-4d68-9bb6-c4b43df75f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:06.941025 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e266bdc2-b668-4d68-9bb6-c4b43df75f36 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:06.941650 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266bdc2-b668-4d68-9bb6-c4b43df75f36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:06.943583 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e266bdc2-b668-4d68-9bb6-c4b43df75f36 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:06.972433 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1433/2888] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:55:06 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:55:06.976707 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dabac7-7769-4455-aac7-5113b5375b63 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:06.988685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7dabac7-7769-4455-aac7-5113b5375b63 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:07.001177 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:07.048823 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bd37d-55c5-4272-9cc1-4278e6d06db1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:07.049422 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd37d-55c5-4272-9cc1-4278e6d06db1 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:07.049699 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd37d-55c5-4272-9cc1-4278e6d06db1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:07.049971 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09bd37d-55c5-4272-9cc1-4278e6d06db1 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:07.050584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bd37d-55c5-4272-9cc1-4278e6d06db1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:07.052502 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09bd37d-55c5-4272-9cc1-4278e6d06db1 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:07.073479 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1456/2889] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:55:06 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:55:07.078171 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266bdc2-b668-4d68-9bb6-c4b43df75f36 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:07.088449 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e266bdc2-b668-4d68-9bb6-c4b43df75f36 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:07.095272 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077d2d-3243-49ff-ab5f-933f619acbba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:07.095867 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077d2d-3243-49ff-ab5f-933f619acbba admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:07.096142 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077d2d-3243-49ff-ab5f-933f619acbba admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:07.096410 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74077d2d-3243-49ff-ab5f-933f619acbba admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:07.097092 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74077d2d-3243-49ff-ab5f-933f619acbba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:07.098733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74077d2d-3243-49ff-ab5f-933f619acbba admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:07.109934 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1434/2890] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:55:07.113893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bd37d-55c5-4272-9cc1-4278e6d06db1 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:07.124463 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09bd37d-55c5-4272-9cc1-4278e6d06db1 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:07.129233 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e5968c-4c83-453b-a22e-2a00cfb4e5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:07.129826 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5968c-4c83-453b-a22e-2a00cfb4e5cf admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:07.130122 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5968c-4c83-453b-a22e-2a00cfb4e5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:07.130386 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e5968c-4c83-453b-a22e-2a00cfb4e5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:07.131065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5968c-4c83-453b-a22e-2a00cfb4e5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:07.132657 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e5968c-4c83-453b-a22e-2a00cfb4e5cf admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:07.144029 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1457/2891] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:55:07.148041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077d2d-3243-49ff-ab5f-933f619acbba admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:07.159543 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74077d2d-3243-49ff-ab5f-933f619acbba admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:07.164157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755497b0-3390-4689-a50d-23a9a5c27a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:07.164819 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755497b0-3390-4689-a50d-23a9a5c27a6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:07.165100 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755497b0-3390-4689-a50d-23a9a5c27a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:07.165373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755497b0-3390-4689-a50d-23a9a5c27a6e admin admin] PATH_INFO: `/v3/projects/8e63ceff0a244e6eb0c2d92a7dfb8886/users/aaf9288adf22418795478fb5a85dee0e/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:07.180024 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755497b0-3390-4689-a50d-23a9a5c27a6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e63ceff0a244e6eb0c2d92a7dfb8886, user_id=aaf9288adf22418795478fb5a85dee0e, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:07.182688 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755497b0-3390-4689-a50d-23a9a5c27a6e admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:07.195564 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1435/2892] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:55:07 2022] PUT /identity/v3/projects/8e63ceff0a244e6eb0c2d92a7dfb8886/users/aaf9288adf22418795478fb5a85dee0e/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:07.206617 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020534-34ee-48d4-b269-3f51e8bfb04c None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:07.206922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020534-34ee-48d4-b269-3f51e8bfb04c None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:07.207191 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8020534-34ee-48d4-b269-3f51e8bfb04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:07.235095 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-e8020534-34ee-48d4-b269-3f51e8bfb04c None None] MFA Rules not processed for user `aaf9288adf22418795478fb5a85dee0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:07.236281 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-e8020534-34ee-48d4-b269-3f51e8bfb04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:07.299057 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1458/2893] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:07 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:07.310992 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755497b0-3390-4689-a50d-23a9a5c27a6e admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:07.312733 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-755497b0-3390-4689-a50d-23a9a5c27a6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:07.370046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755497b0-3390-4689-a50d-23a9a5c27a6e admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:07.374140 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9049734e-3d22-4b9a-b32b-0b5d0232809d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd2da9e23277a4de2a61b9727737ab8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:07.374735 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049734e-3d22-4b9a-b32b-0b5d0232809d service cinder] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:07.375013 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049734e-3d22-4b9a-b32b-0b5d0232809d service cinder] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:07.375283 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9049734e-3d22-4b9a-b32b-0b5d0232809d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:07.383503 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049734e-3d22-4b9a-b32b-0b5d0232809d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:07.384958 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9049734e-3d22-4b9a-b32b-0b5d0232809d service cinder] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:07.393761 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1436/2894] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:07 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:55:14.449959 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8020534-34ee-48d4-b269-3f51e8bfb04c None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:14.512862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8020534-34ee-48d4-b269-3f51e8bfb04c None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:14.517265 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:14.517883 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:14.518158 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:14.518427 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] PATH_INFO: `/v3/users/aaf9288adf22418795478fb5a85dee0e` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:14.519530 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aaf9288adf22418795478fb5a85dee0e)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:14.521494 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:14.552109 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:14.571208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aaf9288adf22418795478fb5a85dee0e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:14.578751 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aaf9288adf22418795478fb5a85dee0e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:14.579363 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aaf9288adf22418795478fb5a85dee0e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:14.591190 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aaf9288adf22418795478fb5a85dee0e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:14.598090 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aaf9288adf22418795478fb5a85dee0e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:14.603837 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1459/2895] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:55:14 2022] DELETE /identity/v3/users/aaf9288adf22418795478fb5a85dee0e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:14.608869 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a470037-3a9f-4f81-81cc-d516deb7fad6 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:14.609349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a470037-3a9f-4f81-81cc-d516deb7fad6 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:14.609466 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a470037-3a9f-4f81-81cc-d516deb7fad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:14.628162 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-4a470037-3a9f-4f81-81cc-d516deb7fad6 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:14.629373 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4a470037-3a9f-4f81-81cc-d516deb7fad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:14.646078 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:14.689737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1437/2896] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:14.699229 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:14.701214 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:14.759743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27d5fdb-655a-4330-8f07-40026e0a8a88 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:14.764258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911988b-24b3-44c4-9e79-e7fe07388ce3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:14.764818 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911988b-24b3-44c4-9e79-e7fe07388ce3 service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:14.765102 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911988b-24b3-44c4-9e79-e7fe07388ce3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:14.765374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911988b-24b3-44c4-9e79-e7fe07388ce3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:14.776359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911988b-24b3-44c4-9e79-e7fe07388ce3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:14.778208 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911988b-24b3-44c4-9e79-e7fe07388ce3 service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:14.788309 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1460/2897] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:55:15.060828 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a470037-3a9f-4f81-81cc-d516deb7fad6 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:15.062334 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4a470037-3a9f-4f81-81cc-d516deb7fad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:15.077463 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a470037-3a9f-4f81-81cc-d516deb7fad6 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:15.082306 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:15.083487 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:15.083932 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:15.084220 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] PATH_INFO: `/v3/projects/8e63ceff0a244e6eb0c2d92a7dfb8886` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:15.085403 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e63ceff0a244e6eb0c2d92a7dfb8886)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:15.087422 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:15.134634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e63ceff0a244e6eb0c2d92a7dfb8886', 'request_id': 'req-6aae3707-e891-4add-96ec-22ac81f11b73'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:15.141312 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e63ceff0a244e6eb0c2d92a7dfb8886', 'request_id': 'req-6aae3707-e891-4add-96ec-22ac81f11b73'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:15.148510 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e63ceff0a244e6eb0c2d92a7dfb8886', 'request_id': 'req-6aae3707-e891-4add-96ec-22ac81f11b73'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:15.159172 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] The token cache is being invalidate because project 8e63ceff0a244e6eb0c2d92a7dfb8886 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81665) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:55:15.159460 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:15.160085 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-6aae3707-e891-4add-96ec-22ac81f11b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:55:15.161634 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1438/2898] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:55:15 2022] DELETE /identity/v3/projects/8e63ceff0a244e6eb0c2d92a7dfb8886 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:15.166877 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426e939-bb0b-4f35-8add-f8f7748aa995 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:15.167163 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426e939-bb0b-4f35-8add-f8f7748aa995 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:15.167442 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5426e939-bb0b-4f35-8add-f8f7748aa995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:15.187813 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-5426e939-bb0b-4f35-8add-f8f7748aa995 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:15.189059 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-5426e939-bb0b-4f35-8add-f8f7748aa995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:15.247368 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1461/2899] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:15.256111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6dfa-261a-4ba8-8158-0bb9158609e4 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:15.256392 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6dfa-261a-4ba8-8158-0bb9158609e4 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:15.256664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6dfa-261a-4ba8-8158-0bb9158609e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:15.276144 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-056a6dfa-261a-4ba8-8158-0bb9158609e4 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:15.277357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-056a6dfa-261a-4ba8-8158-0bb9158609e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:15.285346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1439/2900] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:15.289525 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426e939-bb0b-4f35-8add-f8f7748aa995 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:15.300015 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5426e939-bb0b-4f35-8add-f8f7748aa995 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:15.304587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25640a85-f045-4ec2-9778-5ba74d5f418a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:15.305172 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25640a85-f045-4ec2-9778-5ba74d5f418a admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:15.305451 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25640a85-f045-4ec2-9778-5ba74d5f418a admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:15.305722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25640a85-f045-4ec2-9778-5ba74d5f418a admin admin] PATH_INFO: `/v3/domains` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:15.306367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25640a85-f045-4ec2-9778-5ba74d5f418a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81666) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 06 22:55:15.306668 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25640a85-f045-4ec2-9778-5ba74d5f418a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:15.308306 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25640a85-f045-4ec2-9778-5ba74d5f418a admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:15.320584 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1462/2901] 10.222.0.39 () {58 vars in 1153 bytes} [Tue Sep 6 22:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 06 22:55:15.324953 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a6dfa-261a-4ba8-8158-0bb9158609e4 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:15.334133 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a6dfa-261a-4ba8-8158-0bb9158609e4 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:15.338766 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2507796-2bc8-42e6-a48e-76930dc66c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:15.339349 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2507796-2bc8-42e6-a48e-76930dc66c99 admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:15.339621 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2507796-2bc8-42e6-a48e-76930dc66c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:15.339888 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2507796-2bc8-42e6-a48e-76930dc66c99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:15.340509 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2507796-2bc8-42e6-a48e-76930dc66c99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:15.342412 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2507796-2bc8-42e6-a48e-76930dc66c99 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:15.370418 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1440/2902] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:55:15 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:55:15.374408 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25640a85-f045-4ec2-9778-5ba74d5f418a admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:15.384258 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25640a85-f045-4ec2-9778-5ba74d5f418a admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:15.416545 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:15.442804 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61602072-a59c-45a2-be0c-4bfe13ee5f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:15.443403 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61602072-a59c-45a2-be0c-4bfe13ee5f49 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:15.443680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61602072-a59c-45a2-be0c-4bfe13ee5f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:15.443951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61602072-a59c-45a2-be0c-4bfe13ee5f49 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:15.444577 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61602072-a59c-45a2-be0c-4bfe13ee5f49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:15.446516 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61602072-a59c-45a2-be0c-4bfe13ee5f49 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:15.468175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1463/2903] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:55:15 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:55:15.472361 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2507796-2bc8-42e6-a48e-76930dc66c99 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:15.482739 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2507796-2bc8-42e6-a48e-76930dc66c99 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:15.487551 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4e994-208b-4364-9171-0347a28dabb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:15.488249 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4e994-208b-4364-9171-0347a28dabb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:15.488535 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4e994-208b-4364-9171-0347a28dabb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:15.488833 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d4e994-208b-4364-9171-0347a28dabb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:15.489470 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4e994-208b-4364-9171-0347a28dabb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:15.491058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d4e994-208b-4364-9171-0347a28dabb7 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:15.501439 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1441/2904] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:55:15.505135 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61602072-a59c-45a2-be0c-4bfe13ee5f49 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:15.514951 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61602072-a59c-45a2-be0c-4bfe13ee5f49 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:15.519618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f845f16-a44a-4911-92fe-f97a53fe1bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:15.520204 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f845f16-a44a-4911-92fe-f97a53fe1bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:15.520483 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f845f16-a44a-4911-92fe-f97a53fe1bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:15.520775 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f845f16-a44a-4911-92fe-f97a53fe1bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:15.521411 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f845f16-a44a-4911-92fe-f97a53fe1bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:15.522993 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f845f16-a44a-4911-92fe-f97a53fe1bb5 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:15.533448 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1464/2905] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:55:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 06 22:55:15.537432 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4e994-208b-4364-9171-0347a28dabb7 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:15.548375 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d4e994-208b-4364-9171-0347a28dabb7 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:15.553121 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc2724-8074-478b-b311-c40a10c266fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:15.553774 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc2724-8074-478b-b311-c40a10c266fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:15.554051 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc2724-8074-478b-b311-c40a10c266fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:15.554323 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc2724-8074-478b-b311-c40a10c266fe admin admin] PATH_INFO: `/v3/projects/9b9a2326d5c84ea8bcfc43d1ec1f1908/users/8dbf669e41524ef6a696bef77dab7080/roles/aac9ca64da09463ba9b3966c7188d7ee` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:15.569019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc2724-8074-478b-b311-c40a10c266fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b9a2326d5c84ea8bcfc43d1ec1f1908, user_id=8dbf669e41524ef6a696bef77dab7080, role_id=aac9ca64da09463ba9b3966c7188d7ee)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:15.571600 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc2724-8074-478b-b311-c40a10c266fe admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:15.585262 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1442/2906] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:55:15 2022] PUT /identity/v3/projects/9b9a2326d5c84ea8bcfc43d1ec1f1908/users/8dbf669e41524ef6a696bef77dab7080/roles/aac9ca64da09463ba9b3966c7188d7ee => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:15.591361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26a530-30f4-433e-87d8-2cfc73615247 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:15.591361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26a530-30f4-433e-87d8-2cfc73615247 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:15.591598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f26a530-30f4-433e-87d8-2cfc73615247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:15.612338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-7f26a530-30f4-433e-87d8-2cfc73615247 None None] MFA Rules not processed for user `fee2283327204c6f8ee811e2c566e379`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:15.613598 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-7f26a530-30f4-433e-87d8-2cfc73615247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:15.671795 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1465/2907] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:15.681357 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc2724-8074-478b-b311-c40a10c266fe admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:15.683010 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-b6cc2724-8074-478b-b311-c40a10c266fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:15.728094 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:15.741411 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc2724-8074-478b-b311-c40a10c266fe admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:15.745042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cad599-f596-4486-ae5b-2913ed939dfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:15.745646 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cad599-f596-4486-ae5b-2913ed939dfd service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:15.745924 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cad599-f596-4486-ae5b-2913ed939dfd service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:15.746192 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cad599-f596-4486-ae5b-2913ed939dfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:15.755342 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cad599-f596-4486-ae5b-2913ed939dfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:15.756781 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cad599-f596-4486-ae5b-2913ed939dfd service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:15.766315 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1443/2908] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:55:19.713018 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefc0ea-3314-4d1c-94bf-4df50b1776cf None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:19.713449 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefc0ea-3314-4d1c-94bf-4df50b1776cf None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:19.713594 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cefc0ea-3314-4d1c-94bf-4df50b1776cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:19.743876 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-2cefc0ea-3314-4d1c-94bf-4df50b1776cf None None] MFA Rules not processed for user `8dbf669e41524ef6a696bef77dab7080`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:19.745315 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2cefc0ea-3314-4d1c-94bf-4df50b1776cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:19.816583 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1466/2909] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:19.821060 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cad599-f596-4486-ae5b-2913ed939dfd service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:19.834281 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cad599-f596-4486-ae5b-2913ed939dfd service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:19.841000 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb23fa-4dfe-4d3f-860c-cdaaf5eadbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:19.841783 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb23fa-4dfe-4d3f-860c-cdaaf5eadbfd admin admin] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:19.842155 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb23fa-4dfe-4d3f-860c-cdaaf5eadbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:19.842532 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cb23fa-4dfe-4d3f-860c-cdaaf5eadbfd admin admin] PATH_INFO: `/v3/projects` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:19.843286 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb23fa-4dfe-4d3f-860c-cdaaf5eadbfd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:19.845438 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cb23fa-4dfe-4d3f-860c-cdaaf5eadbfd admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:19.878366 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1444/2910] 10.222.0.39 () {60 vars in 1153 bytes} [Tue Sep 6 22:55:19 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 06 22:55:19.882524 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefc0ea-3314-4d1c-94bf-4df50b1776cf None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:19.894086 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cefc0ea-3314-4d1c-94bf-4df50b1776cf None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:19.942302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:19.958901 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476a280-e448-4bbc-9910-5b1b40835932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:19.959632 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476a280-e448-4bbc-9910-5b1b40835932 admin admin] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:19.960056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476a280-e448-4bbc-9910-5b1b40835932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:19.960361 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4476a280-e448-4bbc-9910-5b1b40835932 admin admin] PATH_INFO: `/v3/users` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:19.961030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476a280-e448-4bbc-9910-5b1b40835932 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:19.962947 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4476a280-e448-4bbc-9910-5b1b40835932 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:19.987065 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1467/2911] 10.222.0.39 () {60 vars in 1144 bytes} [Tue Sep 6 22:55:19 2022] POST /identity/v3/users => generated 299 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 06 22:55:19.991111 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb23fa-4dfe-4d3f-860c-cdaaf5eadbfd admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:20.002153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cb23fa-4dfe-4d3f-860c-cdaaf5eadbfd admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:20.006737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4556bc-c46d-45ec-827b-7a2093575f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:20.007333 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4556bc-c46d-45ec-827b-7a2093575f29 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:20.007605 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4556bc-c46d-45ec-827b-7a2093575f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:20.007870 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4556bc-c46d-45ec-827b-7a2093575f29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:20.008531 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4556bc-c46d-45ec-827b-7a2093575f29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:20.010153 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4556bc-c46d-45ec-827b-7a2093575f29 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:20.021314 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1445/2912] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:55:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:55:20.025482 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476a280-e448-4bbc-9910-5b1b40835932 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:20.036595 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4476a280-e448-4bbc-9910-5b1b40835932 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:20.041329 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fca88c-376e-405f-89e5-30400c3b9f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:20.042030 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fca88c-376e-405f-89e5-30400c3b9f53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:20.042338 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fca88c-376e-405f-89e5-30400c3b9f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:20.042596 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fca88c-376e-405f-89e5-30400c3b9f53 admin admin] PATH_INFO: `/v3/projects/c960541a5d9d4e749752d803dadd9cef/users/f6bcaa2c2a0c431cbcdd6459be0b6248/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:20.060083 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fca88c-376e-405f-89e5-30400c3b9f53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c960541a5d9d4e749752d803dadd9cef, user_id=f6bcaa2c2a0c431cbcdd6459be0b6248, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:20.062763 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fca88c-376e-405f-89e5-30400c3b9f53 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:20.077043 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1468/2913] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:55:20 2022] PUT /identity/v3/projects/c960541a5d9d4e749752d803dadd9cef/users/f6bcaa2c2a0c431cbcdd6459be0b6248/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:20.081093 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4556bc-c46d-45ec-827b-7a2093575f29 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:20.092337 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4556bc-c46d-45ec-827b-7a2093575f29 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:20.148853 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0624e62-8f6c-4805-a9bc-553c755a0e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:20.149450 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0624e62-8f6c-4805-a9bc-553c755a0e06 admin admin] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:20.149722 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0624e62-8f6c-4805-a9bc-553c755a0e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:20.149989 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0624e62-8f6c-4805-a9bc-553c755a0e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:20.150611 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0624e62-8f6c-4805-a9bc-553c755a0e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:20.152192 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0624e62-8f6c-4805-a9bc-553c755a0e06 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:20.162851 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1446/2914] 10.222.0.39 () {58 vars in 1122 bytes} [Tue Sep 6 22:55:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 06 22:55:20.167008 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fca88c-376e-405f-89e5-30400c3b9f53 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:20.177888 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fca88c-376e-405f-89e5-30400c3b9f53 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:20.182350 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bbcc3-c6be-4e37-ac5f-b2410bc26437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:20.182984 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bbcc3-c6be-4e37-ac5f-b2410bc26437 admin admin] REQUEST_METHOD: `PUT` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:20.183252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bbcc3-c6be-4e37-ac5f-b2410bc26437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:20.183515 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80bbcc3-c6be-4e37-ac5f-b2410bc26437 admin admin] PATH_INFO: `/v3/projects/c960541a5d9d4e749752d803dadd9cef/users/f6bcaa2c2a0c431cbcdd6459be0b6248/roles/7a917eb93a9d4f8187269ed155e8e387` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:20.185779 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bbcc3-c6be-4e37-ac5f-b2410bc26437 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c960541a5d9d4e749752d803dadd9cef, user_id=f6bcaa2c2a0c431cbcdd6459be0b6248, role_id=7a917eb93a9d4f8187269ed155e8e387)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:20.188359 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80bbcc3-c6be-4e37-ac5f-b2410bc26437 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:20.198164 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1469/2915] 10.222.0.39 () {60 vars in 1483 bytes} [Tue Sep 6 22:55:20 2022] PUT /identity/v3/projects/c960541a5d9d4e749752d803dadd9cef/users/f6bcaa2c2a0c431cbcdd6459be0b6248/roles/7a917eb93a9d4f8187269ed155e8e387 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:55:24.663346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306fde4-c5fd-414f-b917-3b7539cc18eb None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:24.663832 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306fde4-c5fd-414f-b917-3b7539cc18eb None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:24.663937 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306fde4-c5fd-414f-b917-3b7539cc18eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:24.731268 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-c306fde4-c5fd-414f-b917-3b7539cc18eb None None] MFA Rules not processed for user `f6bcaa2c2a0c431cbcdd6459be0b6248`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:24.732608 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-c306fde4-c5fd-414f-b917-3b7539cc18eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:24.839036 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1447/2916] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:24.853193 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bbcc3-c6be-4e37-ac5f-b2410bc26437 admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:24.854904 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-a80bbcc3-c6be-4e37-ac5f-b2410bc26437 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:24.939715 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80bbcc3-c6be-4e37-ac5f-b2410bc26437 admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:24.943453 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b0591-767e-428a-8a84-b5ccb4c3ce9d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42cf265450454ff9a27a29093336205e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:24.944345 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b0591-767e-428a-8a84-b5ccb4c3ce9d service glance] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:24.944618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b0591-767e-428a-8a84-b5ccb4c3ce9d service glance] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:24.944935 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1b0591-767e-428a-8a84-b5ccb4c3ce9d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:25.012314 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b0591-767e-428a-8a84-b5ccb4c3ce9d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:25.013913 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1b0591-767e-428a-8a84-b5ccb4c3ce9d service glance] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:25.023614 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1470/2917] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:24 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:55:25.163463 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d59fd-c8e5-4077-9eb7-6389b2c61e32 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:25.163743 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d59fd-c8e5-4077-9eb7-6389b2c61e32 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:25.164041 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d59fd-c8e5-4077-9eb7-6389b2c61e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:25.216244 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:25.233005 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-1c1d59fd-c8e5-4077-9eb7-6389b2c61e32 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:25.234139 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-1c1d59fd-c8e5-4077-9eb7-6389b2c61e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:25.297482 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1448/2918] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:55:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:25.347234 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b0591-767e-428a-8a84-b5ccb4c3ce9d service glance] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:25.348648 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-fc1b0591-767e-428a-8a84-b5ccb4c3ce9d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:25.384127 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:25.443568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1b0591-767e-428a-8a84-b5ccb4c3ce9d service glance] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:25.448849 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4b211-d27a-4b3b-8d65-358fea74fc89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:25.450196 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4b211-d27a-4b3b-8d65-358fea74fc89 service swift] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:25.450473 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4b211-d27a-4b3b-8d65-358fea74fc89 service swift] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:25.450747 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4b211-d27a-4b3b-8d65-358fea74fc89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:25.459056 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4b211-d27a-4b3b-8d65-358fea74fc89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:25.460618 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da4b211-d27a-4b3b-8d65-358fea74fc89 service swift] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:25.468643 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1471/2919] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:55:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:55:34.187592 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df03ed-4a99-4fcf-8cf0-fe4de222c0f5 None None] REQUEST_METHOD: `POST` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:34.188103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df03ed-4a99-4fcf-8cf0-fe4de222c0f5 None None] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:34.188103 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31df03ed-4a99-4fcf-8cf0-fe4de222c0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:34.207519 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.auth.core [None req-31df03ed-4a99-4fcf-8cf0-fe4de222c0f5 None None] MFA Rules not processed for user `8dbf669e41524ef6a696bef77dab7080`. Rule list: `[]` (Enabled: `True`). {{(pid=81665) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:34.208700 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-31df03ed-4a99-4fcf-8cf0-fe4de222c0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:34.216187 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1449/2920] 10.222.0.39 () {58 vars in 958 bytes} [Tue Sep 6 22:55:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:34.485934 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4b211-d27a-4b3b-8d65-358fea74fc89 service swift] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:34.487324 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-3da4b211-d27a-4b3b-8d65-358fea74fc89 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:34.545082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4b211-d27a-4b3b-8d65-358fea74fc89 service swift] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:34.548892 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2806a-00eb-4fc4-9e23-885de9c6dd6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:34.549492 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2806a-00eb-4fc4-9e23-885de9c6dd6a service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:34.549766 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2806a-00eb-4fc4-9e23-885de9c6dd6a service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:34.550035 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2806a-00eb-4fc4-9e23-885de9c6dd6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:34.551875 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-81e2806a-00eb-4fc4-9e23-885de9c6dd6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:34.568711 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2806a-00eb-4fc4-9e23-885de9c6dd6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:34.570237 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2806a-00eb-4fc4-9e23-885de9c6dd6a service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:34.580082 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1472/2921] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:55:36.177150 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df03ed-4a99-4fcf-8cf0-fe4de222c0f5 None None] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:36.178618 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-31df03ed-4a99-4fcf-8cf0-fe4de222c0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:36.256231 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31df03ed-4a99-4fcf-8cf0-fe4de222c0f5 None None] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:36.261289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21364b7a-8398-4409-bb60-99ee042ab99e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:36.262055 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364b7a-8398-4409-bb60-99ee042ab99e service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:36.262337 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364b7a-8398-4409-bb60-99ee042ab99e service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:36.262617 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364b7a-8398-4409-bb60-99ee042ab99e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:36.264416 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-21364b7a-8398-4409-bb60-99ee042ab99e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:36.275631 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21364b7a-8398-4409-bb60-99ee042ab99e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:36.277298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21364b7a-8398-4409-bb60-99ee042ab99e service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:36.287253 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1450/2922] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:55:37.483537 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e6199-87a8-407c-86e9-e7470a6033d9 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:37.484050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e6199-87a8-407c-86e9-e7470a6033d9 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:37.484261 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443e6199-87a8-407c-86e9-e7470a6033d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:37.504303 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-443e6199-87a8-407c-86e9-e7470a6033d9 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:55:37.505685 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-443e6199-87a8-407c-86e9-e7470a6033d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:37.513787 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1473/2923] 10.222.0.39 () {58 vars in 1009 bytes} [Tue Sep 6 22:55:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:55:37.526087 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21364b7a-8398-4409-bb60-99ee042ab99e service placement] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:37.537533 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21364b7a-8398-4409-bb60-99ee042ab99e service placement] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:37.542400 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb6302-7e9e-440e-91e4-7617a7db1588 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:37.543019 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb6302-7e9e-440e-91e4-7617a7db1588 service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:37.543298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb6302-7e9e-440e-91e4-7617a7db1588 service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:37.543561 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb6302-7e9e-440e-91e4-7617a7db1588 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:37.551750 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb6302-7e9e-440e-91e4-7617a7db1588 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:37.553367 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb6302-7e9e-440e-91e4-7617a7db1588 service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:37.563732 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1451/2924] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:55:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:55:45.225731 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e6199-87a8-407c-86e9-e7470a6033d9 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:45.236137 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443e6199-87a8-407c-86e9-e7470a6033d9 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:45.239834 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd5dc2-6d5f-46d4-b491-a2486f70470f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:45.240429 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd5dc2-6d5f-46d4-b491-a2486f70470f service neutron] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:45.240704 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd5dc2-6d5f-46d4-b491-a2486f70470f service neutron] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:45.241004 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd5dc2-6d5f-46d4-b491-a2486f70470f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:45.242710 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-2cbd5dc2-6d5f-46d4-b491-a2486f70470f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:45.253181 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd5dc2-6d5f-46d4-b491-a2486f70470f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:45.254581 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd5dc2-6d5f-46d4-b491-a2486f70470f service neutron] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:45.262638 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1474/2925] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:55:47.559781 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=70 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:50.450379 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb6302-7e9e-440e-91e4-7617a7db1588 service swift] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:50.452038 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-4aeb6302-7e9e-440e-91e4-7617a7db1588 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:50.542622 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb6302-7e9e-440e-91e4-7617a7db1588 service swift] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:50.549840 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d35811-a9f3-4991-898b-2784f104fb6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b16947313dad4d59a97cc8b016ee54c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'ResellerAdmin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:50.550655 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d35811-a9f3-4991-898b-2784f104fb6b service nova] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:50.551067 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d35811-a9f3-4991-898b-2784f104fb6b service nova] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:50.551447 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d35811-a9f3-4991-898b-2784f104fb6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:50.553848 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-f5d35811-a9f3-4991-898b-2784f104fb6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:50.572537 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d35811-a9f3-4991-898b-2784f104fb6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:50.574245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d35811-a9f3-4991-898b-2784f104fb6b service nova] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:50.585602 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1452/2926] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:55:55.258830 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=62 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:55:57.087038 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd5dc2-6d5f-46d4-b491-a2486f70470f service neutron] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:55:57.104669 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd5dc2-6d5f-46d4-b491-a2486f70470f service neutron] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:55:57.111254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa7c17-ee48-44a8-a28b-8f75786d0d99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:55:57.111843 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa7c17-ee48-44a8-a28b-8f75786d0d99 service placement] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:55:57.112121 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa7c17-ee48-44a8-a28b-8f75786d0d99 service placement] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:55:57.112387 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa7c17-ee48-44a8-a28b-8f75786d0d99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:55:57.114100 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-29aa7c17-ee48-44a8-a28b-8f75786d0d99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:55:57.126319 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa7c17-ee48-44a8-a28b-8f75786d0d99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:55:57.127868 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa7c17-ee48-44a8-a28b-8f75786d0d99 service placement] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:55:57.138226 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1475/2927] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:55:57 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:56:00.581499 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=34 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:56:01.968514 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d35811-a9f3-4991-898b-2784f104fb6b service nova] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:56:01.977753 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d35811-a9f3-4991-898b-2784f104fb6b service nova] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:56:01.981921 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3dded-1b1d-4cdb-8d55-a7306a5e20bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2658435f74a947f39e207465ebff22d8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:56:01.982501 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3dded-1b1d-4cdb-8d55-a7306a5e20bb service neutron] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:56:01.982775 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3dded-1b1d-4cdb-8d55-a7306a5e20bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:56:01.983042 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c3dded-1b1d-4cdb-8d55-a7306a5e20bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:56:01.991737 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3dded-1b1d-4cdb-8d55-a7306a5e20bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:56:01.993183 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c3dded-1b1d-4cdb-8d55-a7306a5e20bb service neutron] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:56:02.002298 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1453/2928] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:56:01 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:56:07.133431 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=6 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:56:11.998256 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=6 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:57:16.234236 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c55c05-e983-44b1-bc75-02f3abdd09b5 None None] REQUEST_METHOD: `POST` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:57:16.234767 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c55c05-e983-44b1-bc75-02f3abdd09b5 None None] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:57:16.234767 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c55c05-e983-44b1-bc75-02f3abdd09b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:57:16.255562 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.auth.core [None req-57c55c05-e983-44b1-bc75-02f3abdd09b5 None None] MFA Rules not processed for user `7da74f16cb9b4b15a902947103a25cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81666) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 06 22:57:16.256743 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-57c55c05-e983-44b1-bc75-02f3abdd09b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:57:16.264970 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1476/2929] 10.222.0.39 () {58 vars in 967 bytes} [Tue Sep 6 22:57:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 06 22:57:16.304894 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3dded-1b1d-4cdb-8d55-a7306a5e20bb service neutron] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:57:16.317068 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c3dded-1b1d-4cdb-8d55-a7306a5e20bb service neutron] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:57:16.322194 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3b2ec-1c2d-49d7-8002-0da6ae7036c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3f142a9e141440cbc854ade4f4135a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:57:16.322788 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3b2ec-1c2d-49d7-8002-0da6ae7036c5 service swift] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:57:16.323058 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3b2ec-1c2d-49d7-8002-0da6ae7036c5 service swift] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:57:16.323957 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e3b2ec-1c2d-49d7-8002-0da6ae7036c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:57:16.332289 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3b2ec-1c2d-49d7-8002-0da6ae7036c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:57:16.333876 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e3b2ec-1c2d-49d7-8002-0da6ae7036c5 service swift] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:57:16.342046 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1454/2930] 10.222.0.39 () {60 vars in 1384 bytes} [Tue Sep 6 22:57:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 06 22:57:22.642896 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c55c05-e983-44b1-bc75-02f3abdd09b5 None None] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:57:22.644302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-57c55c05-e983-44b1-bc75-02f3abdd09b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:57:22.662130 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c55c05-e983-44b1-bc75-02f3abdd09b5 None None] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:57:22.669425 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941110a8-980e-4570-b0e9-e3356dcb7dcc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:57:22.670031 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941110a8-980e-4570-b0e9-e3356dcb7dcc service placement] REQUEST_METHOD: `GET` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:57:22.670302 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941110a8-980e-4570-b0e9-e3356dcb7dcc service placement] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:57:22.670568 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941110a8-980e-4570-b0e9-e3356dcb7dcc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:57:22.672367 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-941110a8-980e-4570-b0e9-e3356dcb7dcc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:57:22.694433 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941110a8-980e-4570-b0e9-e3356dcb7dcc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:57:22.696356 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941110a8-980e-4570-b0e9-e3356dcb7dcc service placement] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:57:22.706245 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3b2ec-1c2d-49d7-8002-0da6ae7036c5 service swift] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:57:22.710374 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1477/2931] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 22:57:22 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 22:57:22.774745 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e3b2ec-1c2d-49d7-8002-0da6ae7036c5 service swift] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:57:22.779664 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:57:22.780288 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:57:22.780600 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:57:22.781152 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] PATH_INFO: `/v3/users/8dbf669e41524ef6a696bef77dab7080` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:57:22.782267 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=8dbf669e41524ef6a696bef77dab7080)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:57:22.784255 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:57:22.829762 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8dbf669e41524ef6a696bef77dab7080'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:22.830403 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8dbf669e41524ef6a696bef77dab7080'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:22.837360 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8dbf669e41524ef6a696bef77dab7080'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:22.844489 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8dbf669e41524ef6a696bef77dab7080'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:22.858452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8dbf669e41524ef6a696bef77dab7080'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:22.864520 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1455/2932] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:57:22 2022] DELETE /identity/v3/users/8dbf669e41524ef6a696bef77dab7080 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 06 22:57:23.033675 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941110a8-980e-4570-b0e9-e3356dcb7dcc service placement] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:57:23.046960 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941110a8-980e-4570-b0e9-e3356dcb7dcc service placement] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:57:23.105963 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:57:23.106587 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:57:23.106866 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:57:23.107139 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] PATH_INFO: `/v3/projects/9b9a2326d5c84ea8bcfc43d1ec1f1908` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:57:23.108252 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b9a2326d5c84ea8bcfc43d1ec1f1908)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:57:23.110175 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:57:23.156893 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b9a2326d5c84ea8bcfc43d1ec1f1908', 'request_id': 'req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.164330 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b9a2326d5c84ea8bcfc43d1ec1f1908', 'request_id': 'req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.170382 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b9a2326d5c84ea8bcfc43d1ec1f1908', 'request_id': 'req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.180973 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] The token cache is being invalidate because project 9b9a2326d5c84ea8bcfc43d1ec1f1908 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:57:23.181254 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.181872 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.183817 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1478/2933] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:57:23 2022] DELETE /identity/v3/projects/9b9a2326d5c84ea8bcfc43d1ec1f1908 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:57:23.187893 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:57:23.189346 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:57:23.202724 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038ff4d9-b2eb-410e-ab71-5d8426fca7ca admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:57:23.256536 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:57:23.257182 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:57:23.257457 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:57:23.257726 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] PATH_INFO: `/v3/users/f6bcaa2c2a0c431cbcdd6459be0b6248` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:57:23.258827 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6bcaa2c2a0c431cbcdd6459be0b6248)` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:57:23.261570 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:57:23.266452 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:57:23.308549 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6bcaa2c2a0c431cbcdd6459be0b6248'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.309216 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6bcaa2c2a0c431cbcdd6459be0b6248'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.316806 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6bcaa2c2a0c431cbcdd6459be0b6248'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.324079 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6bcaa2c2a0c431cbcdd6459be0b6248'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.335734 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.notifications [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6bcaa2c2a0c431cbcdd6459be0b6248'} {{(pid=81665) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.341868 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1456/2934] 10.222.0.39 () {58 vars in 1224 bytes} [Tue Sep 6 22:57:23 2022] DELETE /identity/v3/users/f6bcaa2c2a0c431cbcdd6459be0b6248 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:57:23.791326 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] Authenticating user token {{(pid=81666) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 22:57:23.793793 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.fernet_utils [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81666) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 22:57:23.806680 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e5f7cb-c531-4ab8-adbe-18f29ddeaa4e admin admin] Validating token access rules against request {{(pid=81666) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 22:57:23.863927 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fee2283327204c6f8ee811e2c566e379', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '693d7b3554de4d2c9e0e732722fe61d1', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81666) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 22:57:23.864549 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 22:57:23.864862 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 22:57:23.865134 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] PATH_INFO: `/v3/projects/c960541a5d9d4e749752d803dadd9cef` {{(pid=81666) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 22:57:23.866249 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c960541a5d9d4e749752d803dadd9cef)` {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 22:57:23.868131 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] RBAC: Authorization granted {{(pid=81666) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 22:57:23.879384 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:57:23.915250 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c960541a5d9d4e749752d803dadd9cef', 'request_id': 'req-ed82e8db-4a48-4a23-b0fb-57cfce155a89'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.922616 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c960541a5d9d4e749752d803dadd9cef', 'request_id': 'req-ed82e8db-4a48-4a23-b0fb-57cfce155a89'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.928050 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c960541a5d9d4e749752d803dadd9cef', 'request_id': 'req-ed82e8db-4a48-4a23-b0fb-57cfce155a89'} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.938646 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] The token cache is being invalidate because project c960541a5d9d4e749752d803dadd9cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81666) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 06 22:57:23.938922 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.939542 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG keystone.notifications [None req-ed82e8db-4a48-4a23-b0fb-57cfce155a89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81666) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 06 22:57:23.941023 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: [pid: 81666|app: 0|req: 1479/2935] 10.222.0.39 () {58 vars in 1233 bytes} [Tue Sep 6 22:57:23 2022] DELETE /identity/v3/projects/c960541a5d9d4e749752d803dadd9cef => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 06 22:57:33.338237 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 22:57:33.935722 dv-cinder-852208-20-iscsi devstack@keystone.service[81666]: DEBUG dbcounter [-] [81666] Writing DB stats keystone:SELECT=19,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81666) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 06 23:01:01.048287 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] Authenticating user token {{(pid=81665) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 06 23:01:01.049691 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 23:01:01.188178 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da886dbe-6eeb-4fb5-93f4-4139532ca733 admin admin] Validating token access rules against request {{(pid=81665) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 06 23:01:01.251238 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ccbb07-5b11-480d-9164-cfcd64ef0d35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1d7e1b7dfa32448a88e0c90f10cdb2e9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fbb4b236264d4062a720496ccb7cbbfe', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81665) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 06 23:01:01.251841 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccbb07-5b11-480d-9164-cfcd64ef0d35 service placement] REQUEST_METHOD: `GET` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 06 23:01:01.252119 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccbb07-5b11-480d-9164-cfcd64ef0d35 service placement] SCRIPT_NAME: `/identity` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 06 23:01:01.252389 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ccbb07-5b11-480d-9164-cfcd64ef0d35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81665) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 06 23:01:01.254157 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.fernet_utils [None req-86ccbb07-5b11-480d-9164-cfcd64ef0d35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81665) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 06 23:01:01.265235 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccbb07-5b11-480d-9164-cfcd64ef0d35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 06 23:01:01.266798 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ccbb07-5b11-480d-9164-cfcd64ef0d35 service placement] RBAC: Authorization granted {{(pid=81665) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 06 23:01:01.276481 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: [pid: 81665|app: 0|req: 1457/2936] 10.222.0.39 () {60 vars in 1365 bytes} [Tue Sep 6 23:01:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 06 23:01:11.272454 dv-cinder-852208-20-iscsi devstack@keystone.service[81665]: DEBUG dbcounter [-] [81665] Writing DB stats keystone:SELECT=89 {{(pid=81665) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}